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Abstract 



Over the past three decades policymakers have been seeking new ways to secure 
improved performance from higher education institutions. One popular approach has 
been performance funding, which involves use of a formula to tie funding to institutional 
performance on specified indicators. This report reviews findings from studies on 
performance funding programs in a multitude of states. The studies suggest that tying 
funding to outputs has immediate impacts on colleges in the form of changes in funding, 
greater awareness by institutions of state priorities and of their own institutional 
performance, and increased status competition among institutions. Because of these 
immediate impacts, performance funding produces intermediate institutional changes in 
the form of greater use of data in institutional planning and policymaking and in changes 
in academic and student service policies and practices that promise to improve student 
outcomes. However, claims that performance funding does indeed increase ultimate 
outcomes — in the form of improved rates of retention, completion of developmental 
education, and graduation — are not validated by solid data. In the face of this finding, this 
report identifies obstacles to the effective functioning of performance funding, as well as 
unintended impacts. The report closes by providing recommendations for overcoming the 
many obstacles to the effective functioning of performance funding and addressing the 
unintended impacts documented by the studies reviewed. 
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1. Introduction 



Over the past three decades policymakers have been actively seeking new ways to 
secure improved performance from higher education institutions. One of the more 
popular approaches has been performance funding. In contrast with performance 
reporting or performance budgeting, performance funding uses a clearly specified 
formula to tie funding to institutional performance on indicators such as student retention, 
attainment of certain credit levels, and other student outcomes (Burke, 2002; Dougherty 
& Hong, 2006; Hamisch, 2011). 

Performance funding for higher education has had an up and down history. After 
first appearing in 1979 with Tennessee’s pioneering program, it was enacted by an 
increasing number of states through the 1990s, but then receded in the early 2000s as 
many states gave up their programs (Dougherty & Natow, 2009). Since then, however, it 
has been enjoying a considerable revival. Several states recently enacted or readopted 
performance funding, including Washington and Indiana in 2007, and several other states 
(including Arkansas, Colorado, Illinois, and Massachusetts ) are actively discussing it 
(Harnisch, 2011).' 

Essential questions for state policymakers who are considering performance 
funding are these: What impacts does such funding have on student outcomes, how are 
those impacts produced, and what obstacles and unintended effects are encountered in the 
process? 

This report, prepared by Community College Research Center (CCRC) 
investigators, addresses these questions by reviewing findings from studies on 
performance funding programs in a multitude of states. We focus on Florida, Tennessee, 
and Washington, where we found the most extensive and systematic studies, though we 
also report the results of studies of the impacts of performance funding in other states, 
such as Missouri, North Carolina, Ohio, and South Carolina. 



* Performance funding is also receiving considerable attention from national policy groups and higher 
education associations. The National Conference of State Legislatures and the National Governors 
Association held conferences in 2010 and 2011 that addressed performance funding, and the American 
Association of State Colleges and Universities has issued a policy brief on it (Harnisch, 201 1). 
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2. Research Methods 



2.1 Conceptualization of the Impacts of Performance Funding 

Performance funding programs aim to improve institutional performance, 
particularly with respect to student outcomes. We call these the “ultimate” outcomes 
desired. These include improvements in, for example, student retention, passage of key 
courses, attainment of certain levels of credit accrual, graduation, job placement, and 
responsiveness to labor market demands. These ultimate outcomes show up as the 
performance indicators that performance funding programs use as the basis for allocating 
funds. 

In order to realize these outcomes, performance funding programs embody 
“theories of action” (Argyris & Schon, 1996) as to how these ultimate outcomes are to be 
produced. The most obvious theory of action is that institutional performance will be 
improved through material incentives that mimic the profit motive for businesses 
(Dougherty & Hong, 2006, pp. 59-60).^ In Washington, the legislators who were the 
main proponents of the performance funding program that the state operated between 
1997 and 1999 (see below for more) were described by a state-level higher education 
official as believing “in the notion that we tend to get more of what the funding structure 
responds to, so what is incentivized and measured and funded, we tend to get more of and 
less of other things.” Similarly, in Florida, the leading legislative advocate of 
performance funding was described as believing that “you could get performance altered 
by money. If you put a pot of money out there, people would change their behavior in 
order to chase that money.” 



^ Argyris and Schon (1996, p. 13) note: “The general form of a theory of action is: If you intend to produce 
consequence C in situation S, then do A. Two further elements enter into the general schema of a theory of 
action: the values attributed to C that make it seem desirable as an end-in-view and the underlying 
assumptions, or model of the world, that make it plausible that action A will produce consequence C in 
situation S.” The concept of a theory of action closely parallels that of “policy instruments,” defined as 
“mechanisms that translate substantive policy goals into concrete actions” (McDonnell & Elmore, 1987, p. 
134). 

^ This theory of action closely resembles “inducement” as a policy instrument (McDonnell & Elmore, 

1987, pp. 134, 137-138) or “remuneration” as a source of organizational compliance (Etzioni as cited in 
Matland, 1995, p. 161). Both policy instruments are rooted in a resource dependence perspective; that is, 
that organizations are shaped by the degree and form of their dependence on resources from the external 
environment (Pfeffer & Salancik, 1978; Scott, 1987, p. 111). 
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However, performance funding programs also may invoke — less often and more 
quietly — three other theories of action.^ Two involve the provision of information. 
Performance funding programs (as with performance reporting programs) could improve 
institutional performance by increasing institutions’ awareness of their state’s higher 
educational goals as well as of their own performance with respect to those goals. The 
underlying theory of action is that institutions will be prodded to improve their 
performance, either through persuasion or shame, insofar as they come to realize that 
their institutional performance is not meeting the state’s goals or is not in keeping with 
their own institutional standards (Dougherty & Hong, 2006, pp. 60-61; see also 
Anderson, 2006, pp. 230-231; Ewell, 1999, p. 194).^ The final theory of action involves 
the fomenting of status competition among colleges. Performance funding programs 
could catalyze institutional changes by affecting institutions’ desire to rank well against 
their institutional peers (Dougherty & Hong, 2006, pp. 61-62). 

Changes in state funding for colleges, colleges’ awareness of state priorities and 
of their own performance relative to those priorities, and colleges’ concern about how 
well they are performing relative to peer colleges can be termed the immediate impacts of 
performance funding. To be effective, these immediate changes must catalyze 
intermediate changes involving modifications of institutional policies, programs, and 
practices — such as changes in instruction and student support services — that will result in 
the ultimate outcomes of interest to policymakers, such as more baccalaureate graduates 
or higher job placement rates. Typically, state performance funding programs are quite 
vague about what are the intermediate changes that they are seeking. Hence, one of the 
key tasks of this report is to sketch out what intermediate impacts, relevant to the goals of 
performance funding, do occur. 



These other theories of action may be less often “espoused theories” of action than “theories in use”; that 
is, action sequences that are not accidental but are not necessarily intended. Argyris and Schon note that: 
“By ‘espoused theory’ we mean the theory of action which is advanced to explain or justify a given pattern 
of activity. By ‘theory-in-use’ we mean the theory of action which is implicit in the performance of that 
pattern of activity. A theory-in-use is not a ‘given.’ It must be constructed from observation of the pattern 
of action in question” (1996, p. 13). 

^ This theory of action resembles, but goes beyond, the “hortatory” technique of control described by 
Anderson (2006, pp. 230-231). The particular strand that focuses on increasing institutions’ awareness of 
gaps in their performance relative to their own goals and standards resembles the theory of action described 
by Estela Bensimon, Alicia Dowd, and colleagues in connection with the Equity for All and Community 
College Student Success Projects (Bensimon, 2005; Baldwin, Bensimon, Dowd, & Kleiman, 2011; see also 
Dowd & Tong, 2007). 
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In addition to these different elements of the theories of action underlying 
performance funding, we also need to consider its unintended impacts and frequent 
obstacles (Dougherty & Hong, 2006, pp. 69, 73). The unintended impacts constitute 
outcomes that are not intended by the enacting body, but which arise as side effects of 
funding institutions based on their performance. These can take such forms as the 
weakening of academic standards or the narrowing of institutional missions to those that 
are financially rewarded. The obstacles to the success of performance funding can 
include such things as performance indicators that do not adequately capture institutional 
performance, performance funding not keeping pace with improvements in institutional 
performance, and inequality in institutional capacity to diagnose performance problems 
and determine workable solutions. 

2.2 Data Search 

To locate research that addressed the impacts noted above, we searched for 
studies of the impact of performance funding on higher education institutions by combing 
the following major electronic databases: ERIC, LexisNexis, ProQuest, and ProQuest 
Digital Dissertations. We searched using various combinations of the following search 
terms: 



• higher education, community college, university; 

• performance funding, performance reporting, performance 
budgeting; 

• accountability; and 

• impacts, outcomes, results. 

In addition, we searched the websites of the main state higher education 
coordinating or governing boards in all states known to have had performance funding 
programs. We looked for publications by those boards that addressed the effects of their 
performance funding programs. 

A review of abstracts and summaries helped narrow the body of studies to those 
examining the impacts of performance funding specifically (as opposed to performance 
budgeting or performance reporting). We also eliminated works that — even if they 
addressed the impacts of performance funding — were simply based in the opinion of the 
author and that did not have any more rigorous empirical basis. The final body of 
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literature consisted of 40 studies: four analyzing national data, 16 on Tennessee, nine on 
Florida, five on South Carolina, two each on Ohio and Washington, and one each on 
Missouri and North Carolina. While our studies happened to be clustered around these 
states, we have to emphasize that we aimed to include studies from any state that had 
performance funding.^ 

2.3 Data Analysis 

We coded the findings in these studies using both preestablished categories and 
categories that emerged in the process of analyzing the selected research studies (Miles & 
Huberman, 1994). We began with categories developed by Dougherty and Hong (2006) 
for performance funding impacts (immediate, mediated, and ultimate), obstacles, and 
unintended impacts. These became our “start list.” However, following a “grounded” 
approach, we added new categories in order to capture elements of performance funding 
impacts that emerged in the studies we were reviewing. The results of our data coding 
were independently cross-checked by at least one other person. 

2.4 Limitations 

While there is much to be learned from the research literature on performance 
funding, we also wish to warn readers that there are considerable limitations to that 
literature. There are very few multivariate quantitative studies. The many qualitative 
studies tend to be limited in their scope, often restricting themselves to one state and even 
just to one institution. Moreover, these qualitative studies often rely too much on self- 
reports and do not triangulate these data with other kinds of evidence. We will come back 
to these limitations in the conclusions to this report. 



® The high number of studies on Tennessee seems explicable by a couple of factors. Tennessee established 
the first and longest lasting performance funding program in the country, which alone would raise the 
possibility that more studies would accumulate. Moreover, one of the key architects of that program — 
Professor E. Grady Bogue — is at the University of Tennessee and encouraged many of his doctoral students 
to conduct case studies of the impact of performance funding on different kinds of colleges and 
universities. Florida, meanwhile, has had a long-lasting and well known performance funding program, the 
Performance Based Budgeting program established in 1994. It therefore has lasted long enough to provoke 
study. Moreover, Florida has a highly professionalized state department of education and other state 
agencies, with staffers who have produced several studies of the impact of the performance funding 
program, either in their professional capacities or as part of their doctoral studies. 
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3. Differences among Performance Funding Programs 

Before we launch into our review of the research literature, we should clarify the 
nature of performance funding. We need to distinguish two dominant forms (performance 
funding 1.0 and 2.0) and then note differences between states in the particular versions of 
these two dominant forms that they have enacted. 

3.1 Performance Funding 1.0 and 2.0 

State performance funding programs come in many different forms, but it is 
useful to make a distinction between what has been called performance funding (PF) 1.0 
and 2.0 (Albright, 2009; Snyder, 2011). Performance funding 1.0 takes the form of a 
bonus, over and above regular state funding for higher education. The funding is 
allocated on the basis of certain typical indicators: ultimate outcome indicators such as 
numbers (or percentages) graduating or being placed in jobs; intermediate achievement 
indicators such as retention, developmental education completion, and transfer; and more 
occasionally, input indicators such as enrollments of students of certain backgrounds and 
indicators of program quality such as percentage of licensure exam takers who pass 
(Burke, 2002; Dougherty, Hare, & Natow, 2009). Examples of such PF 1.0 programs are 
the ones established in Tennessee in 1979, Florida in 1994, Ohio in 1995, and 
Washington in 1997 (see below). 

Performance funding 2.0 programs, while not sacrificing indicators of ultimate 
outcomes, also put considerable emphasis on indicators of intermediate achievement: for 
example, course completions; successful completion of developmental education courses 
or programs; passage of key gateway courses such as college mathematics or college 
English; and reaching certain credit thresholds such as 15 or 30 credits (Offenstein & 
Shulock, 2010). The Student Achievement Initiative enacted in Washington in 2007 is a 
notable example of the use of these intermediate achievement indicators (Jenkins, 

Ellwein, & Boswell, 2009; Shulock & Jenkins, 2011). Where PF 2.0 programs take an 
even more radical departure from PF 1.0 is in having performance funding no longer be a 
bonus but rather embedding it in the regular state base funding formula for higher 
education. The new performance funding programs established in Ohio in 2009 and 
Tennessee in 2010 are notable examples; they abandoned enrollment-based funding for 
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the four-year colleges and, in the case of Tennessee, the two-year colleges as well (see 
below for more). 

Three factors have played major roles in the rise of PF 2.0. One has been rising 
doubt about the efficacy of PF 1.0. State officials have begun to question whether basing 
even as much as 5% of state appropriations to higher education on performance outcomes 
is enough to compel significant improvements in institutional practices and student 
outcomes (Sanford & Hunter, 2011). Moreover, there is a growing sense that, due to the 
current and foreseeable stagnation of the economy, future state budgets are unlikely to 
have much leeway to provide performance funding bonuses on top of regular, formula- 
based state funding for higher education. Rather, it is argued, performance indicators will 
have to be built into the basic state formula itself. Finally, the PF 2.0 programs in Ohio, 
Tennessee, and Washington have received endorsements from the U.S. Department of 
Education and national policy groups such as the National Governors Association, the 
National Conference of State Legislatures, Lumina Foundation, the Gates Foundation, 
and others (Albright, 2009; Hamisch, 2011; Lederman, 2008; Snyder, 2011; Sparks & 
Waits, 2011; U.S. Department of Education, 2011).^ Moreover, the Obama 
Administration’s college completion agenda and Race to the Top competition have also 
sharpened interest in performance funding. 

We describe below the performance programs in Elorida, Ohio, Tennessee, and 
Washington because of the preponderance of studies of those states and the considerable 
interest in the PE 2.0 aspects of performance funding in the latter three states. Each of 
these states has had two different performance funding programs (Dougherty, Natow, 
Hare, Jones, & Vega, 2011; Dougherty, Natow, & Vega, 2012). 

3.2 Florida’s Two Performance Funding Programs 

At one point Florida had two performance funding programs operating 
concurrently. Its Performance-Based Budgeting (PBB) program was established in 1994 

^ Dennis Jones of the National Center for Higher Education Management Systems has been making this 
point for some time. 

* Cross-state policy diffusion processes played an important part in the development of PF 1 .0, and their 
impact seems to be even bigger with respect to PF 2.0. Moreover, national policy organizations and policy 
foundations appear to be playing a bigger role in the case of PF 2.0 than of PF 1 .0 (Albright, 2009; 
Dougherty et ah, 201 1 ; Dougherty & Puleio, 2011; Harnisch, 2011; Snyder, 2011; Sparks & Waits, 201 1 ; 
U.S. Department of Education, 2011). 
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and is still in place (although funding was suspended in 2008). However, the Workforce 
Development Education Fund (WDEF), which was established in 1997, lapsed in 2002 
(Dougherty & Natow, 2009; Dougherty et al., 2012). While the PBB is an example of a 
PE 1.0 program, one could argue that the WDEF was an early example of a PE 2.0 
program. 

Performance-Based Budgeting: 1994-present. In 1994, Florida passed the 
Government Performance and Accountability Act, which was intended to move state 
funding of government agencies toward program outcomes rather than inputs. The state’s 
community colleges were some of the first public organizations to come under the 
purview of PBB, which went into effect for them in 1996. The state’s four-year colleges 
and universities were supposed to become subject to PBB some time later, but this never 
happened (Bell, 2005, p. 42; Dougherty et al., 2011; Wright, Dallet, & Copa, 2002, pp. 
144-145; Yancey, 2002, pp. 56-57). 

When the 1994 Government Performance and Accountability Act was passed in 
1994 (Eaws of Florida ch. 94-249), it was envisioned as a form of performance budgeting 
in which the legislature would base its funding for higher education institutions on the 
performance they had already demonstrated and set performance targets for the future. 
There was no fixed formula tying funding to specific indicators and that displeased key 
senators, who wanted such a formula. The legislature thus devised the Performance 
Incentive Funding program in 1996 (1996-97 General Appropriations Act, HB 2715, 
Specific Appropriation 172A). This program, which created a specific pot of money that 
would be allocated by formula to community colleges based on specific performance 
indicators, became a part of Performance-Based Budgeting. 

Initially, $12 million was appropriated for fiscal year 1996-97. These funds were 
to be distributed to community colleges at the end of the fiscal year, depending on their 
individual performance on three sets of indicators. Five million dollars was distributed on 
the basis of how many students graduated (completed certificates, associate of arts, or 
associate of science degrees). Another $5 million was distributed on the basis of the 
performance of specific groups of students: numbers graduating among students who 
were economically disadvantaged, disabled, non-English speakers, or participants in 
English as a Second Fanguage (ESF) programs; numbers who graduated and passed state 
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job licensure exams; and numbers who graduated and secured a job in a targeted 
occupation. The remaining $2 million pertained to time to degree: it was allocated on the 
basis of how many students completed an associate of arts degree with less than 72 
attempted credit hours (Wright et al., 2002, pp. 144-145). 

Over the years, PBB funding has accounted for 1% to 2% of total state 
appropriations for the community colleges. While the performance indicators changed 
over time, they continued to focus on degree completion, transfer to the state university 
system, successful passage of licensure exams, and securing jobs paying more than $10 
an hour (Bell, 2005, pp. 39-43, 53-54; Dougherty & Natow, 2010; Florida State Board 
for Community Colleges, 1998, 2000; Office of Program Policy Analysis and 
Government Accountability, 1997; Wright et al., 2002, pp. 148-149, 161, 163, 165, 250- 
252; Yancey, 2002, pp. 56-58). 

Workforce Development Education Fund: 1997-2002. Initiated in 1997, the 
Workforce Development Education Fund (WDEF) (Eaws of Florida, ch. 93-307, 

SB 1688) applied to the community colleges and area vocational centers operated by the 
K-12 districts. The WDEF operated between 1997-98 and 2000-01 and then lapsed 
(Dougherty & Natow, 2009; Dougherty et al., 2012). At its peak, the WDEF comprised 
nearly 6% of state funding for community colleges. It held back 15% of an institution’s 
state appropriation from the previous year for vocational and technical education. 
Institutions could then win this money back based on their performance on these 
measures: (1) number of adult basic education completions, vocational certificates, and 
vocational associate of science for students with certain characteristics (economically 
disadvantaged, recipients of welfare, disabled, dislocated, and ESF); and (2) job 
placement of students, with institutions getting more points for placement in higher 
paying jobs (Bell, 2005, pp. 47, 59-60, 175-176; Florida State Board for Community 
Colleges, 1998, 2000; Pfeiffer, 1998; Wright et al., 2002, p. 163; Yancey, 2002, pp. 59- 
61). 

3.3 Ohio’s Old and New Performance Funding Programs 

Ohio established two performance funding programs in 1995 and then replaced 
them with a new program established in 2009. The original programs were both examples 
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of PF 1.0; the new program is a leading example of PF 2.0. Because the new formula is 
so recent, the studies we draw on regarding the impact of performance funding in Ohio 
apply only to the earlier system. 

The Success Challenge and Performance Challenge: 1995-2009. In 1995, 

Ohio established the Success Challenge, which provided a bonus to universities based on 
the number of students who earned baccalaureate degrees within four years, with students 
in financial need weighted more (Moden & Williford, 2002, pp. 173-178). Ohio also 
established the Performance Challenge (soon abandoned) which — though largely not a 
performance funding program^ — did reward community colleges, technical colleges, and 
branch campuses based on the number of students who transferred or relocated after 
completing at least 15 quarter hours or 10 semester hours of coursework and on the 
number of transfer or relocated students who completed baccalaureate degrees (Dunlop- 
Loach, 2000, Appendix B; Ohio Board of Regents, 1994).^° 

New funding formula: 2009-present: In 2009, Ohio passed legislation ending 
enrollment-based funding for four-year universities and adding a performance-funding 
element to its funding for two-year colleges. In the process, the Success Challenge was 
terminated. The new state funding formula for the state universities and their regional 
campuses took effect in the 2010-1 1 fiscal year. For the 14 universities in Ohio, the state 
will allocate all funding based on the number of courses and degrees completed by 
students rather than enrollment early in the semester. Initially, funding will favor course 
completion, but over time degree completion will grow in importance. For the 24 
regional campuses, funding initially will be based solely on course completion, with 



^ The other eight service expectations under the Performance Challenge involved additional state support 
for providing broad job training, offering effective developmental education, providing noncredit 
continuing education opportunities, fostering business partnerships, developing high school linkages, 
providing accessible learning environment and effective instructional delivery strategies, keeping tuition 
and fees low, and creating high community involvement (Burke & Serban, 1998, pp. 40-41; Dunlop- 
Loach, 2000, Appendix B; Moden & Williford, 2002, pp. 173-177; Ohio Board of Regents, 1994). 

The other challenges did not address student outcomes. The Research Challenge allocated appropriations 
in proportion to dollars raised from external grants and contracts. The Access Challenge provided money to 
community colleges, technical colleges, university regional campuses, and three four-year “access” 
universities to restrain tuition and fees and operate support programs to raise student recruitment and 
retention. Colleges received funds in proportion to their share of General Studies enrollments across all 
public access institutions. The Jobs Challenge rewarded two-year colleges for their efforts in workforce 
preparation. Colleges got funding in proportion to their share of total qualified expenditures for noncredit 
job related training (Burke & Serban, 1998, pp. 40-41; Moden & Williford, 2002, pp. 173-177; Ohio 
Board of Regents, 1994). 
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degree completion added to the formula over time. In both cases, funding will be 
weighted by the cost of programs and whether students are at risk, defined initially only 
in terms of being eligible for state need-based aid (Ohio Board of Regents, n.d.; Petrick, 
2010). The new funding plan for the community colleges begins in fiscal year 201 1-12. 
The proportion of the state formula allocated on the basis of performance indicators is 
projected to rise from 5% in fiscal year 2012 to 30% in fiscal year 2015. The 
performance indicators include the number of students who successfully complete 
remedial education (on three different measures), the number of students who earn 15 and 
30 semester credit hours, the number who earn an associate degree or transfer to an Ohio 
four-year college or university, and performance on an indicator chosen by community 
colleges themselves (R. Abrams, personal communication, July 19, 2010; Ohio 
Association of Community Colleges, 2010). 

3.4 Tennessee’s Old and New Performance Funding Programs 

Tennessee established its first performance funding program for higher education 
in 1979, a program that exists to this day. It was the pioneering example of PF 1.0. In 
2010, the state established a second performance funding program that will run in tandem 
with the first system. This new program is a leading example of PF 2.0. Because the 
second program has not yet taken effect, the studies on Tennessee that we review here 
focus only on the first program. 

Performance funding: 1979-present: The Tennessee Higher Education 
Commission adopted performance funding for the state’s public two- and four-year 
higher education institutions in 1979 (Banta, 1986; Banta, Rudolph, Van Dyke, & Fisher, 
1996; Bogue, 2002; Dougherty et ah, 2011). Funds were first allocated to institutions 
using performance funding in fiscal year 1980-81. Under that system, higher education 
institutions could earn a bonus of 2% over and above their annual state appropriations for 
achieving certain goals based on five performance indicators, each of which was worth 
20 out of 100 points (Banta et ah, 1996; Bogue & Johnson, 2010; Levy, 1986). The 
original indicators were these: program accreditation (proportion of eligible programs in 
the institution’s inventory that are accredited), student major field performance (student 
performance in major fields as assessed by examinations that have normative standards 
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for state, regional, or national referent groups), student general education performance 
(student performance in general education as assessed by a nationally-normed exam such 
as the ACT-COMP examination), evaluation of instructional programs (evaluative 
surveys of a representative sample of current students, recent alumni, or community 
members or employers), and evaluation of academic programs by peer review teams of 
scholars from institutions outside the state and/or practicing professionals in a field 
(Banta, 1986, pp. 123-128; Bogue, 1980; Bogue & Johnson, 2010). 

Tennessee added nine performance funding indicators and dropped four between 
1979-80 and 2009-10. The changes focused on student achievement in college and 
institutional improvement.^* In addition, the percentage of additional funding that 
institutions could earn based on performance increased from 2% to 5.45% of the base 
state appropriation (Bogue & Johnson, 2010; Dougherty & Natow, 2010). 

Complete College Tennessee program: 2010-present: In 2010, the Tennessee 
legislature passed the Complete College Tennessee Act, part of which provided for the 
dramatic redesign of the basic higher education funding formula. Previously, Tennessee’s 
funding formula had been largely enrollment-driven, meaning that institutions received 
the bulk of their state appropriations based on the number of students enrolled in their 
institutions. The changes made by the Complete College Tennessee Act, however, will 
render the funding formula predominantly outcomes-driven as of the 201 1-12 fiscal year 
(Dougherty & Natow, 2010; Tennessee Higher Education Commission, n.d.a, n.d.b.). 

This new program supplements rather than replaces the 1979 program. It is not a 
bonus to the existing enrollment state funding formula, but rather, revamps the state 
funding formula by shifting its focus from enrollments to persistence and graduation. 
During the first year of the new system’s operation in fiscal year 201 1-12, university 
funding will be based on the following indicators: student accumulation of 24, 48, and 72 
hours of credit; research and service expenditures; degrees awarded (bachelor’s and 
associate, master’s and education specialist, and doctoral and law degrees); degrees per 
full-time equivalent (FTE) student; transfers out with at least 12 credit hours; and six- 



** In addition to dropping some indicators while adding others, Tennessee has also changed the weight 
given to some of the continuing indicators. Over the years it reduced the weight it gave to program 
accreditation, from 20 points to 5; general education assessment, 20 to 15; and graduate performance in 
major fields, 20 to 10 (Bogue & Johnson, 2010). 
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year graduation rate (Tennessee Higher Education Commission, 2011, p. 1). Community 
colleges will be funded based on somewhat different criteria: student accumulation of 12, 
24, and 26 hours of credit; workforce training; dual enrollment students; associate 
degrees and certificates granted; awards per full-time-equivalent enrollments; job 
placements; transfers out with 12 credit hours; and remedial and developmental success. 
In addition, an institution is eligible for a 40% bonus for credit and degree completion for 
low-income and adult students (Tennessee Higher Education Commission, 2011). In 
November 2010, the Tennessee Higher Education Commission officially adopted the new 
funding formula and requested appropriations based on the new outcomes-driven model 
for 2011-12 academic-year funding (Dougherty & Natow, 2010). 

3.5 Washington’s Two Programs: One Abandoned, One Recently Added 

The state of Washington has established two performance funding programs for 
higher education. The first system, which applied to both the public universities and the 
community colleges, operated only between 1997-99 before being allowed to lapse 
(Dougherty & Natow, 2009; Dougherty et ah, 2012). It was a good example of a PE 1.0 
program. The second program, the Student Achievement Initiative, was enacted in 2007 
and applies only to the community colleges (Dougherty et ah, 2011; Jenkins et ah, 2009; 
Washington State Board for Community and Technical Colleges [WSBCTC], 2007). It is 
a leading example of a PE 2.0 program insofar as it puts great emphasis on indicators of 
intermediate achievement, such as completion of developmental education and of college 
mathematics. However, it does not embed performance funding in the base state funding 
formula, as do the new programs in Ohio and Tennessee. 

First PF program: 1997-1999. In 1997 Washington enacted a performance 
funding program of accountability for public institutions of higher education as a proviso 
in its appropriations bill for the 1997-99 biennium. A small percentage of institutions’ 
state appropriations was held back and colleges and universities were required to meet or 
exceed certain levels on certain performance indicators in order to receive full 
appropriations (Dougherty et ah, 2011; Washington State Eegislature, 1997; see also 
Nisson, 2003; Washington State Higher Education Coordinating Board, 1998). The 
performance measures that were adopted for four-year institutions differed from those for 
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two-year institutions. For public four-year colleges, the measures related to: persistence 
and completion rates, “faculty productivity,” time-to-degree efficiency, and “[a]n 
additional measure and goal to be selected by the higher education coordinating board . . . 
in consultation with each institution” (Washington State Legislature, 1997; see also 
Washington State Higher Education Coordinating Board, 1998, p. 2). The performance of 
public two-year colleges was measured based on transfer rates, “[c]ore course completion 
rates,” the hourly earnings of institutions’ occupational program alumni, and time-to- 
degree efficiency (Washington State Legislature, 1997; see also Nisson, 2003). 

The Student Achievement Initiative: 2007-present. Washington’s first 
performance funding program ended in 1999 (Dougherty & Natow, 2009; Dougherty et 
ah, 2012). However, the state reintroduced performance funding for higher education 
eight years later. In 2007, the State Board for Community and Technical Colleges 
established the Student Achievement Initiative (SAI) for Washington’s two-year colleges 
(Dougherty et ah, 2011; Jenkins et ah, 2009; WSBCTC, 2007). Unlike the 1997-99 
performance funding proviso, the State Board’s new performance funding system was 
designed not to withhold any money from institutions, but instead to reward technical and 
community colleges with a small amount of new money when their students reached 
certain outcomes thresholds. Performance indicators included achievement of 
competency in basic skills; pass rates for postsecondary-level mathematics and 
developmental coursework; and the number of degrees, certificates, and college credits 
granted. Under the funding formula, colleges received “achievement points” when their 
students reached certain levels on these indicators. The 2008 fiscal year was a “learning 
year,” during which involved community and technical colleges examined their 
performance on these measures and developed plans to improve. During the 2009 fiscal 
year, institutions began to be rewarded for their performance on the measures (Jenkins et 
ah, 2009; WSBCTC, 2007). 
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4. Immediate Impacts of Performance Funding 

The immediate impacts of performance funding are the direct channels through 
which performance funding programs try to catalyze changes in institutional 
performance. The “theories of action” (Argyris & Schon, 1996) involve creating changes 
in institutional finances, knowledge, and status competition in order to produce 
organizational changes that will yield the student outcomes of interest to policymakers. 

4.1 Changes in Funding 

Changes in institutional funding are typically the main policy instrument that 
policymakers have in mind when they consider performance funding. This theory of 
action is premised on the belief that institutions are revenue maximizers and will make a 
strong effort to improve their performance if a significant enough amount of funding is 
involved (Burke, 2002, pp. 266-272). For the most part, the focus has been on additions 
to existing state funding for higher education, but some performance funding programs 
(such as Florida’s Workforce Development Education Fund and Washington’s 1997-99 
performance funding program) have allowed for institutions to suffer a net reduction in 
state funding if their performance is judged insufficient (Dougherty et ah, 2011). 

Institution-level variations in funding are explicitly discussed by four studies on 
Tennessee, two each on Florida and Ohio, as well as single studies on North and South 
Carolina and Washington. The studies on Tennessee touched on how much impact 
performance funding had on the finances of higher education institutions. For the most 
part, the funds involved were relatively small, and some observers saw little impact on 
the institutions. For example, a leader of a Tennessee institution stated: “I think that 
performance funding is incredibly weak. . . . The state subsidy that comes as a result of 
whatever few points you get or not get is not significant enough to make us do what we 
ought to be doing” (as quoted in Tanner, 2005, p. 83). Conversely, a top official of 
another institution noted: “The first year Tennessee Tech got, as a result of the scores on 
performance, something over $700,000. It’s not a lot of money, but it was like manna 
from Heaven and that’s money we would not have had” (as quoted in Lorber, 2001, p. 
82). 
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The Ohio Board of Regents (2008) reported that the overall state allocation to the 
Success Challenge, which rewarded public four-year colleges for producing more 
baccalaureate graduates, grew from $2 million in 1998 to roughly $50 million in 2005/^ 
Two research studies on Ohio noted that the money associated with the Success 
Challenge was perceived as small (for example, Miami University and Cleveland State 
University each received about $3 million). However, it was viewed as a worthwhile 
addition to the regular base funding from the state that had some impact on the 
institutions (O’Neal, 2007, pp. 119, 144-145; Schaller, 2004, p. 80). 

In Florida, funding for individual institutions through the Workforce 
Development Education Fund rose or fell each year on average 4.36% over the period 
1998-2003 (Bell, 2005, pp. 149-151; see also Dougherty & Hong, 2006, p. 59).^"^ There 
are varying reports on how much impact these funding swings had. Dougherty and Hong 
(2006) reported that the colleges did not see these shifts as having much impact. 
However, the state WDEF funding for Florida Community College at Jacksonville 
(FCCJ) decreased $3.4 million between fiscal years 2001 and 2002, which amounted to 
8.7% of its FY 2001 state vocational education appropriation (Bell, 2005, p. 150). The 
college administration described this loss as “serious” and stated that FCCJ “must 
improve performance in this increasingly competitive environment” (as quoted in Morris, 
2002, pp. 130, 145). 

Finally, in Washington, during the early years of the Student Achievement 
Initiative, community colleges each received $51,000 in 2007-08, $66,000 in 2008-09, 
and an average of $60,000 in 2009-10 and another $60,000 in 2010-1 1 (Shulock & 
Jenkins, 2011, p. 6). 

4.2 Increased Awareness of State Priorities 

Besides causing shifts in state funding for colleges, performance funding can 
stimulate organizational responsiveness through another policy instrument: increasing 
institutional awareness of state goals for higher education (Dougherty & Hong, 2006, pp. 
60-61; Jenkins et ah, 2009, pp. 3-4, 19). We found several studies (four on Tennessee 

The report does not report institution-level variations in funding. 

This program operated between 1997 and 2002 (Dougherty & Natow, 2009; Dougherty et al., 2012). 

This figure excludes Palm Beach College for one year, when it had an increase of 129.15% between 
1998-99 and 1999-00 (Bell, 2005, p. 149). Positive changes dominated negative ones: 65% versus 39%. 
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and one each on Florida, Ohio, North Carolina, and Washington) that give some evidence 
that institutional leaders were aware of the details of their state’s performance funding 
program. In Florida, a study of three community colleges found wide awareness in two of 
the colleges of the state mandates linked to performance and the fact that community 
colleges were expected to focus on performance (Bell, 2005, pp. 122, 126-128). In North 
Carolina, the top administrative leaders at four community colleges evidenced a good 
awareness of the terms of the state’s performance funding program (Harbour & Nagy, 
2005, pp. 452-457). In Ohio, an administrator at a public four-year university stated that 
key student affairs personnel were aware of the state’s Success Challenge performance 
funding program because she telephoned them once a year to ask about the programs they 
were operating “to help on retention graduation rate for all students and specifically for 
students who would be considered low SES, low-income, at-risk in terms of what high 
school they come from, being first generation, that sort of thing” (O’Neal, 2007, pp. 126- 
127). 

While there is evidence that college leaders are aware of the terms of state 
performance funding programs, there is also evidence that this awareness is not widely 
diffused throughout institutions. As one moves down the chain of authority, knowledge 
about state performance funding drops considerably. We will come back to this point in 
the section below on obstacles to effective implementation of performance funding. 

4.3 Increased Awareness of Institution’s Own Performance 

Another means by which state performance funding programs attempt to improve 
institutional performance is by increasing colleges’ self-awareness of their own 
institutional outcomes and how these outcomes compare to state standards and 
institutional goals (Burke, 2002, pp. 271-272; Dougherty & Hong, 2006, p. 61). The 
impact of performance funding on institutional self-awareness was discussed in 10 
studies on Tennessee; three on Florida; and one each on Missouri, North Carolina, and 
Ohio. 

In Tennessee, a number of respondents at Volunteer State Community College 
listed self-evaluation and introspection as an upshot to performance funding (Freeman, 
2000, p. 102). For example, one respondent noted: “[The state performance funding 
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policy] does force institutions to look at themselves. ... It presents the information to the 
institution as well as in some instances to the public.” At Tennessee Technological 
University, the Dean of the College of Business stated that the policy forces evaluation of 
whether programs are actually meeting goals or just talking about it (Lorber, 2001, p. 68). 

In Florida, a study of the responses of four community colleges to the state’s 
Performance-Based Budgeting program found that officials at all four noted ways in 
which the state’s performance funding system had led their colleges to become more self- 
reflective about their student’s outcomes and the causes of those outcomes. For example, 
one president noted: “The college has greater internal institutional evaluation. Faculty 
and staff are concerned more with student performance and their completion. We now put 
an emphasis on faculty orientation as it relates to student services, completion, and 
program development” (as quoted in Poisel, 1998, p. 89; also see Bell, 2005, p. 132). 

In North Carolina, one community college made performance on state measures 
an agenda item at departmental meetings and incorporated the measures into the college’s 
independent benchmark reporting process (Harbour & Nagy, 2005, p. 454). At another 
community college, the dean for research and planning asserted that state data had been 
helpful in highlighting the relatively poor performance of the college’s students on real- 
estate licensure exams, and the dean for transfer indicated that state data confirmed a 
need for more resources for the university transfer program (Harbour & Nagy, 2005, p. 
457 ).^^ 

4.4 Increased Status Competition 

State performance funding programs can also catalyze institutional action by 
provoking status competition among colleges according to their performance on outcome 
measures (Dougherty & Hong, 2006, pp. 61-62). Six studies on Tennessee and one each 
on Florida, Missouri, Ohio, and Washington discuss this policy lever. For example, a vice 
president of a Washington community college described the state’s initiative as partially 
having to do with “bragging rights,” because college presidents are competitive people 
who fret over the loss of every performance point (Jenkins et ah, 2009, p. 39). A number 

However, the president and chief instructional officer argued that the state performance funding program 
held colleges accountable for matters they had little control over and they therefore doubted that the 
program would lead to any changes at the college in the near future. 
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of administrators at Tennessee Technological University described performance funding 
as an opportunity to showcase “many of the fine things that Tennessee Tech was doing” 
(as quoted in Lorber, 2001, p. 70) but some administrators at the University of Tennessee 
at Knoxville decried a “hidden agenda” to compare institutions with each other (Hall, 
2000, p. 96). In Missouri, a medium-prestige institution viewed performance funding as a 
way to showcase its strengths and shed “popular perceptions of mediocrity,” while 
administrators at the University of Missouri felt that not complying with the program 
would make it look like the university did not care about learning (as quoted in 
Naughton, 2004, pp. 78, 88-89). 



5. Intermediate Impacts: Changes in Organizational Policies and Practices 

Intermediate impacts consist of organizational changes — made in response to the 
immediate pressures of budget changes, increased awareness of state education goals and 
institutional performance, and heightened status competition — that are designed to boost 
a college’s student outcomes (Dougherty & Hong, 2006, p. 62). Broadly, these 
organizational changes can be broken down into increased use of data in institutional 
planning, improvements in academic policies and practices, and changes to student 
services. 

5.1 Increased Use of Data in Institutional Planning 

A survey of college officials (ranging from presidents to department chairs) in 
five states with performance funding asked them to rate their use of performance results 
in various areas on a five point scale: 1 (very extensively), 2 (extensively), 3 
(moderately), 4 (minimally), 5 (not at all). The average ratings for various areas of 
institutional action ranged between moderate and extensive: institutional planning (2.66), 
student outcomes assessment (2.79), internal budget allocations (3.05), curriculum 
planning (3.08), administrative services (3.34), student services (3.39), and academic 
advising (3.48) (Burke, 2002, p. 72).^*’ 

Interestingly, the community college officials indicated significantly more use of performance data than 
did their four-year college counterparts (Burke, 2002, pp. 72-74). 
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A total of five studies (three each on Tennessee and Florida and one on Missouri) 
explicitly cite how state performance funding programs have led institutions to make 
greater use of data in institutional planning. A study drawing on interviews with 60 top 
campus officials at 14 Tennessee two-year colleges concluded that five of those 
campuses “definitely” and five “probably” have achieved the state goal of “assessment 
based improvements that have fostered more comprehensive and responsive college 
planning processes through building an assessment infrastructure on each college 
campus” (Kastner, 2000, pp. 31, 35; see also Banta & Fisher, 1984 regarding the 
University of Tennessee at Knoxville). Similarly, interviews with about 15 state and local 
community college officials in Florida revealed a common perception that, with the 
advent of state performance funding, data were now playing a larger role in driving local 
decision making (Dowd, 2003, pp. 132-133). In fact, a state survey of Florida community 
colleges and area vocational-technical schools reported that the institutions usually have 
moved to develop new internal data collection systems, provide training for 
administrative staff in data collection, and hire information systems specialists to handle 
the increase in data collection (Gray, Harkreader, & Wagar, 2001, p. 36). 

5.2 Academic Improvements 

Studies covering five states report changes in academic practices in response to 
state performance funding demands. Eight studies on Tennessee; five on Florida; two on 
Washington; and one each on Missouri, North Carolina, and South Carolina describe 
these academic changes. We classify them into three categories, moving from the more 
macro to the more micro: alterations to academic departments, alterations to academic 
programs and curricula, and alterations to courses. 

Alterations to academic department organization and staffing. Alterations to 
academic departments in organizational structures or staffing patterns are reported in 
three studies on Tennessee, two on Florida, and one study each on North Carolina and 
South Carolina. For example, a Florida institution increased its number of associate of 
science programs offered in order to increase performance funding points (Morris, 2002, 
pp. 136, 154). In South Carolina, performance funding that was based in part on class 
sizes, student/faculty ratios, and availability of faculty outside class prompted several 
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colleges to reduce the number of part-time faculty, require full-time faculty to teach 
larger courses, and pay part-time faculty to hold office hours (Fore, 1998, pp. 1 17-1 19). 
Further, in Tennessee, the University of Tennessee at Knoxville reported that its College 
of Education responded to performance funding by replacing traditional departments with 
smaller, faculty-defined units (Hall, 2000, p. 87), and at Tennessee Technological 
University, 17 majors were reduced to four in one undergraduate academic unit and eight 
different graduate programs were consolidated into two (Lorber, 2001, p. 68). 

Alterations to academic programs and curricula. Even when departments were 
not reorganized, changes were made within them in response to performance funding. 
Changes to programs and curricula were reported in seven studies on Tennessee, in four 
on Elorida, in two on Washington, and in single studies on North Carolina and South 
Carolina. 

One alteration involved shutting down programs on efficiency grounds. Evidence 
of such a reaction appears in three Elorida studies and in one each on South Carolina and 
Washington. In Elorida, programs that produced few graduates or whose degrees did not 
lead to jobs on Elorida’ s Targeted Occupations Eist were shut down. Specifically, 
programs that had enrollments of under 10 students were cut unless they were consistent 
sources of points for other reasons, such as the allied health field in which licensure 
standards more or less guaranteed job placement (Bell, 2005, p. 104; Gray et al., 2001, 
pp. 34-36; Morris, 2002, pp. 142-143). Similarly, in South Carolina, two community 
colleges reported eliminating programs and courses with traditionally low enrollments 
(Eore, 1998, p. 118). 

Another alteration to programs and curricula involves changing requirements so 
that students are more likely to graduate and thus generate performance points. In 
Washington, in order to qualify for achievement points, community colleges eliminated 
requirements that impede graduation, such as certificate or diploma fees and increased 
credit requirements for certain shorter occupational certificate programs (Jenkins et ah, 
2009, p. 32). In Elorida, community colleges removed graduation obstacles, such as 
courses that students found hard to pass, and defined intermediate points on the way to a 
degree that could qualify for partial performance points (Dougherty & Hong, 2006, pp. 

65, 73-74). 
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Alterations to course instruction. Five studies on Tennessee, two on Florida, 
and one each on Missouri, North Carolina, and Washington document alterations to 
course content, instructional technique, and testing in order to improve institutional 
performance. For example, several departments at the University of Tennessee developed 
new senior comprehensive exams, course sequences, and course contents (Banta & 

Fisher, 1984, pp. 34-36), and the University of Memphis created faculty groups to study 
ways to improve undergraduate instruction and implement major field tests (Latimer, 
2001, p. 78). In Florida, Valencia Community College established a program to provide 
faculty with course release to work in industry with the aim of making vocational course 
content more relevant and increasing job placement rates (Bell, 2005, p. 121). Also, 
Indian River Community College began monitoring course withdrawal rates and 
encouraging faculty members with high course withdrawal rates to work with more 
successful faculty members to increase retention (Bell, 2005, pp. 129, 132). In 
Washington, two community colleges moved to improve student performance in 
mathematics courses by providing support for additional developmental math course 
offerings and release time for faculty to develop developmental math success strategies 
(Jenkins et ah, 2009, p. 30). 

5.3 Student Services Improvements 

In order to improve their performance in response to performance funding 
demands, colleges and universities have also made changes in their student services. 
Specifically, institutions have looked to improve their services with regard to registration, 
financial aid, first-year retention programs, counseling and advising, tutoring and 
supplemental services, and job placement services. 

Registration and graduation procedures. Two studies on Tennessee and one 
each on Florida and Washington documented modifications to registration procedures as 
a way to improve student satisfaction (a performance funding indicator in Tennessee) and 
remove impediments to re-registration as a way to bolster retention. At the University of 
Tennessee at Knoxville, each dean designated a representative to assist students with 
registration problems during the add/drop period, and the university moved up its 
admissions cut-off date in order to allow more time to assess course demand and create 



22 




additional course sections if needed (Banta & Fisher, 1984, p. 36). At Walters State 
Community College, registration procedures were changed to make it easier for student 
to register for subsequent semesters (Shaw, 2000, p. 91). 

Financial aid. Two Florida studies and one each on Ohio, Tennessee, and 
Washington present evidence of changes to financial aid procedures designed to increase 
student retention and graduation. In Ohio, Miami University increased financial aid for 
students from low-income families and offered tuition exemptions for students with a 
family income under than $35,000 (O’Neal, 2007, p. 125). In addition to increasing 
available aid, institutions moved to strea ml ine and simplify financial aid procedures. A 
Florida institution reorganized its financial aid department to better serve student needs 
(Poisel, 1998, p. 90). 

Retention programs for first-year students. Two studies on Ohio and one each 
on Florida and Washington reported that colleges developed programs to provide better 
support for first-year students. Santa Fe College in Florida created a mandatory “student 
success” course for first-year students. The course focused on working with an advisor, 
selecting a major, and developing study skills (Bell, 2005, p. 107). In Washington, one 
community college responded to the state’s Student Achievement Initiative by creating a 
student success center providing “one-stop” advising, admissions, registration, and 
financial aid services; another redesigned the advising program for first-year students; 
and two others appointed a student achievement coordinator to facilitate student 
transition from pre-college to college-level coursework (Jenkins et ah, 2009, p. 30). 

Counseling/advising. Changes in counseling and advising practices are a 
frequent response to performance funding demands, as indicated by four studies on 
Florida; three on Tennessee; two on Ohio; and single studies on North Carolina, South 
Carolina, and Washington. 

In Florida, some community colleges developed computerized systems to monitor 
student progress and alert campus officials who could intervene to help students who 
were falling behind in their coursework or failing to attend classes (Gray et ah, 2001, p. 
36). For example, at Indian River Community College, students falling behind in meeting 
course requirements are sent a note and called, and counselors are notified so that they 
can intervene before the students fail (Bell, 2005, p. 130). University of Tennessee at 
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Knoxville revamped its advising system after a survey of dropouts — taken in response to 
the state’s performance funding program — revealed that nearly a quarter of the 
respondents stated that they left the university because they could not get advice on 
courses and programs (Banta & Moffett, 1987, p. 40). 

Tutoring and other supplemental instruction. Improvements in tutoring and 
other supplemental instruction in response to state performance funding demands were 
discussed in two studies on Ohio and in one study each on Florida, Tennessee, and 
Washington. At Walters State Community College in Tennessee, student and alumni 
surveys used in part for the state’s performance funding program led administrators to 
realize that students at the college’s satellite campuses needed more tutoring services 
(Shaw, 2000, p. 91). At Florida Community College at Jacksonville, efforts were made to 
provide tutoring to prepare students for GED exams through computer-based instruction 
(Morris, 2002, p. 195). In Washington, three of 17 community colleges studied responded 
to the Student Achievement Initiative by putting more funding into tutoring, including 
support for a tutoring center at one college and support for an online tutoring service at 
another college (Jenkins et ah, 2009, p. 30). 

Job placement services. Improvements in job placement services are discussed 
in two Tennessee studies, three Florida studies, and one Washington study. The 
University of Tennessee at Knoxville responded to performance funding pressures by 
developing more internships and increasing faculty involvement in job placement (Banta 
& Moffett, 1987, p. 40). In Florida, a state survey of workforce education administrators 
at community colleges and area vocational schools in Florida found that many 
respondents reported helping students obtain high-skills, high-wage jobs because the state 
Workforce Development Education Eund (WDEE) performance funding program gave 
greater weight to such jobs. Under the WDEE, students who completed programs and got 
jobs in their field garnered 10 times as many performance funding points as did students 
who completed programs but were not able to get related jobs (Bell, 2005, p. 120; Gray et 
ah, 2001, pp. 34, 36; Morris, 2002, p. 172). Hence, colleges moved to improve their 
career counseling. Elorida colleges surveyed businesses to see how pleased they were 
with graduates. Also, some institutions established placement offices or hired additional 
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counselors with business expertise to assist students with career transition (Gray et al., 
2001, pp. 34, 36). 



6. Ultimate Impacts: Student Outcomes 

The ultimate student outcomes that states are attempting to realize include 
improved rates of retention, successful remediation, credit accrual, transfer, graduation, 
and job placement (Dougherty & Hong, 2006, pp. 62-69; Offenstein & Shulock, 2010).^’ 
Unfortunately, despite the importance of these outcomes, the current research literature 
on performance funding sheds relatively little light on the extent to which they are being 
realized. 

6.1 Improved Retention 

A study of Tennessee four-year institutions found that they did not improve their 
retention rates over the years 1995-03 and that they did not improve on their peer 
institutions in other states that did not have performance funding (Sanford & Hunter, 

201 1). This study controlled for differences among the institutions in enrollments, 
Carnegie classification, tuition revenue as a share of total operating revenues, proportion 
of institutional expenditures going to instruction, and student composition (proportion 
who were minority students, attended part time, or received Pell grants) (Sanford & 
Hunter, 2011). 

In Ohio, a survey of 224 administrators at 13 Ohio public universities yielded 11 
responses citing higher retention rates as a positive outcome of the state’s Success 
Challenge performance funding program (Schaller, 2004, p. 144). However, these self- 
reports are not backed up by any corroborating data, particularly data controlling for 
competing causes of rising retention rates, beyond the presence of performance funding. 



A distinction can be made between ultimate student outcomes, such as graduation, job placement, and 
high scores on tests of general education, and intermediate student outcomes, such as retention, completion 
of remediation, passage of gatekeeper courses, attainment of certain levels of credit accrual, and successful 
transfer (see Offenstein & Shulock, 2010). 
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6.2 Improved Remediation 

Improved rates of remedial success were cited in a study of performance funding 
in Florida and Washington. In both cases, rates of remedial success improved, whether 
measured in terms of percentage of remedial students passing the highest remedial course 
within two years (Florida) or the percentage of English as a Second Language (ESL), 
Adult Basic Education (ABE), and GED students who gained one competency level in at 
least one subject area during the year (Washington) (Dougherty & Hong, 2006, pp. 62- 
64). However, these data do not control for other possible causes of this improvement in 
remediation rates. 

6.3 Improved Graduation Rates 

Data pertaining to changes in graduation rates that might result from performance 
funding come from four studies of Elorida and one each of Washington and Ohio. The 
Ohio Board of Regents found possible positive impacts of the Success Initiative on 
graduation rates, noting that its advent in 1995 was followed by an increase in the number 
of baccalaureate degrees awarded at Ohio public universities. Despite the fact that tuition 
at the university’s main campuses was increasing by about 9% per year during these 
years, the total number of baccalaureate degrees increased 18% between fiscal years 1998 
and 2007 (Ohio Board of Regents, 2008, pp. 10-1 1). Similarly, in Elorida, the number of 
associate degrees and certificates awarded rose 19% in fiscal year 1996-97, just before 
the installation of its new performance funding system, and in 2002-03, when the main 
performance funding program (the WDEE) was ended (Dougherty & Hong, 2006, pp. 
64-65; see also Bell, 2005, pp. 153-15; Gray et ah, 2001, p. 36; Phillips, 2002, pp. 50- 
54). And in Washington, the number of associate degrees and certificate graduates 
jumped 38% between 1996-97, when the state’s first performance funding program was 
enacted, and 2003-04 (Dougherty & Hong, 2006, p. 65). Moreover, the number 
completing an associate degree, certificate, or apprenticeship jumped 22% between 
2006-07, when the state’s second performance funding system was established, and 
2009-10 (WSBCTC, 2010). 

However, these studies quite properly caution against claiming that such jumps in 
the number of graduates are attributable in whole or even in part to performance funding. 
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given the myriad other factors that could be at work (Dougherty & Hong, 2006, p. 69; 
Ohio Board of Regents, 2008, p. 5). One of the most important is that a rising number of 
graduates could simply be due to rising enrollments. In fact, a Florida study (Phillips, 
2002) notes the danger of relying on simple trend lines to establish the impact of 
performance funding. Using data for 1990-91, 1991-92, 1993-94, and 1994-95, Phillips 
developed an equation to predict the number of associate degrees and certificates granted 
per year in subsequent years. The predictors were associate degrees and certificates 
granted the previous year; full-time equivalent (FTE) enrollments in professional courses, 
vocational courses, and college prep (remedial) courses; and the number of non-resident 
aliens, American Indian/ Alaskan Natives, Asian/Pacific Islanders, Hispanics, and White 
non-Hispanics enrolled. The analysis was designed to determine if the actual number of 
degrees and certificates granted in the years 1996-97 to 2000-01 deviated significantly 
from the number predicted by the model. The analysis found a statistically significant 
increase above the trend line for associate of arts degrees but not for associate of science 
degrees or certificates. Even in the case of the increase in associate of arts degrees, this 
rise could have been due to factors that, while associated with performance funding, did 
not really involve improvements in institutional performance: reductions in the length of 
time to earn an associate of arts degree at many institutions, making a degree easier to 
attain; and financial penalties for students not completing courses in a timely fashion 
(Phillips, 2002, pp. 50-67). 

Underscoring the importance of being skeptical about findings regarding the 
impact of performance funding on graduation figures is the fact that five multivariate 
studies (one of Tennessee and four of the United States as a whole) have found no 
statistically significant positive impact of performance funding on six-year graduation 
rates at public four-year colleges (Eryar, 2011; Sanford & Hunter, 2011; Shin, 2010; Shin 
& Milton, 2004; Volkwein & Tandberg, 2008).^* Three of the four studies of the United 
States as a whole drew on federal IPEDS data and controlled for several different 



** Shin (2010) found that states with performance funding (PF) had statistically significant (at the p < 0.10 
level) higher graduation rates. However, he discounted this impact because the addition of variables for 
performance funding and performance budgeting added nothing to total variance explained (Shin, 2010, p. 
59). In addition, Fryar found a marginally significant (p < 0.10) negative impact of performance funding on 
graduation rates. The other studies found positive impacts, but none was statistically significant. 
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institutional characteristics, with two of these also controlling for both institutional and 
state-level characteristics (Shin, 2010; Shin & Milton, 2004). 



7. Obstacles to the Effectiveness of Performance Funding 

The limitations (in both extent and quality) of the data available on student 
outcomes bolster the pervasive counterargument of many institutional officials that 
performance funding has had little real impact on institutional performance and that it is 
largely a symbolic practice. Three studies on Tennessee and one on North Carolina cite 
several administrators and faculty members who argue that performance funding has 
simply been a rote activity, with actors only going through the motions of collecting data 
and submitting reports. For example, the faculty senate chairman at one North Carolina 
community college dismissed performance funding as “mere paper shuffling” (Harbour 
& Nagy, 2005, pp. 457-458). A department chair at the University of Tennessee at 
Knoxville argued: 

The impact that this has had on us in the department has 
really been to simply add another administrative task. I 
don’t think . . . that it has changed the way a single faculty 
member teaches, the kind of materials that a single faculty 
member presents. It has had no impact on our curriculum. 

(As quoted in Hall, 2000, pp. 78-79) 

This pervasive undercurrent of skepticism about performance funding reflects the 
many obstacles that it encounters when implemented. Among the obstacles that crop up 
in the research literature on the impacts of performance funding are concerns about the 
appropriateness of the performance measures employed; instability in funding, indicators, 
and measures; the brief duration of many performance funding programs; funding levels 
that are too low and not well enough insulated against the ups and downs of the state 
revenue cycle; shortfalls in regular state funding for higher education; lack of a clear 
connection for faculty between performance and funding; inequalities in institutional 
capacity; unequal distribution of knowledge and expertise about performance funding 
within institutions; and “game -playing” by institutions. These obstacles are discussed 
below. 
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7.1 Inappropriate Performance Funding Measures 

Many studies discuss the reservations held by administrators and faculty about 
how well different indicators and their measures capture the real performance of their 
institutions. In fact, a faculty leader at one North Carolina community college dismissed 
performance indicators in North Carolina as “pretend” measures (Harbour & Nagy, 2005, 
p. 458). 

Learning assessments. In the eight studies of Tennessee and one each of North 
Carolina and South Carolina, institutional officials expressed skepticism about the 
validity of the learning assessments being used as measures of institutional performance. 
In Tennessee, respondents were particularly skeptical that the state-mandated assessment 
of general education and the major field exit standards adequately captured what faculty 
aimed to teach (Banta & Fisher, 1984, p. 34; Freeman, 2000, p. 98; Hall, 2000, pp. 95- 
96; Shaw, 2000, pp. 88-89; Tanner, 2005, p. 85; Williams, 2005, p. 93). 

Retention and graduation rates. Three studies each on Florida and Tennessee, 
and one on Ohio, raised concerns regarding retention and graduation rates. Community 
college officials and faculty in Florida asked why colleges should be penalized if 
vocational students leave college without a degree for a well-paying job during times of 
economic growth. The community college officials and faculty argued that the student 
must have been able to reach their goals or at least have attained a useful level of 
education (Bell, 2005, p. 109; Gray et ah, 2001, p. 32; Morris, 2001, p. 131; see also 
Dougherty & Hong, 2006, pp. 70-71). Furthermore, graduation rates usually do not take 
transfer into account; thus, community colleges with students who successfully transfer to 
a four-year college without having first received an associate degree usually cannot count 
such students as graduates. In fact, transfer students are often mistakenly treated as if they 
were dropouts (Freeman, 2000, p. 97; O’Neal, 2007, pp. 135-136). Finally, some studies 
raise the issue that graduation rates do not take into account differences between 
institutions in the academic preparation or degree ambitions of students. For example, 
community colleges tend to have higher percentages of students with social and academic 
disadvantages that make it hard to get a degree, even if this is the students’ intent 
(Dougherty & Hong, 2006, pp. 70-71). 
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A problem has also been noted with the use of numbers graduating rather than 
rates of graduation. A college could increase its numbers graduating, even if the 
graduation rate is declining, if it is experiencing sizable enrollment increases (Jenkins et 
al., 2009; Shulock & Jenkins, 2011, p. 10). 

Job placement. Concerns over job placement indicators cropped up in two 
studies each on Tennessee and Florida and one on Washington. A major criticism by 
college officials and faculty was that job placement rates are dependent on the state of the 
local economy, which varies over time and by region, in ways that are not under the 
control of the colleges (Banta et al., 1996, p. 32; Bell, 2005, p. 137; Dougherty & Hong, 
2006, p. 71). In addition, concern was raised in Florida that job placement indicators hurt 
institutions whose graduates obtain jobs out of state, because those jobs are not counted 
by state datasets (Gray et al., 2001, p. 40). 

Institutional differences. One study each on Florida, Missouri, Ohio, South 
Carolina, Tennessee, and Washington noted the concern of college officials about how 
state performance funding programs failed to take into account differences among 
institutions in their mission and in their capacity to meet performance demands. With 
regard to mission, tensions arose in Tennessee over perceptions that the performance 
funding program insufficiently acknowledged that institutions have different missions. 
For example, a university administrator argued that the research mission of that 
institution was not reflected in its performance funding results (Hall, 2000, p. 95). 
Meanwhile, Washington community colleges with a greater focus on academic transfer 
argue that they have been at a disadvantage because the highest potential for amassing 
performance points under the new Student Achievement Initiative occurs in adult basic 
education and developmental education (Jenkins et al., 2009, p. 37). 

With regard to capacity to meet performance demands, Washington community 
colleges serving greater numbers of at-risk students perceive themselves to be at a 
disadvantage in amassing performance points because these students tend to need “costly 
wrap-around services” in order to succeed (Jenkins et al., 2009, p. 37). Similarly, in 
South Carolina, the state Legislative Audit Council found that: 

The standardization of measures for schools in each sector 
raises opposition by institutional representatives. The 
measures do not fully take into account the differences that 
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exist among institutions within a sector. For example, a 
majority of the same measures have been applied to MUSC 
[Medical University of South Carolina] and Clemson when 
they have radically different student populations. (South 
Carolina Legislative Audit Council, 2001, p. 23) 

Similar sentiments about performance funding programs’ disregard of differences among 
institutions in their student bodies and therefore performance capacities arose as well in 
Missouri and Ohio (Naughton, 2004, pp. 89-90; O’Neal, 2007, pp. 130, 137; also see 
Dougherty & Hong, 2006, pp. 71, 73). 

7.2 Instability in Funding, Indicators, and Measures 

When budgets and indicators are unstable, higher education leaders find it hard to 
decide where to focus the efforts of their institutions and they are afraid to take chances. 

A survey in the late 1990s of community college and four-year college officials in five 
states with performance funding found that 40.1% rated budget instability as an extensive 
or very extensive problem of performance funding in their state (Burke, 2002, p. 77). In 
Florida, a dean of vocational education at a Florida community college noted: “Allow us 
to know what the rules are so that we can plan appropriately. I think that the indecision 
each year has really put us in a predicament that has strapped us for resources.” 

As it happens, in Florida, funding for the state’s Performance-Based Budgeting 
program fluctuated over the years. It started at 2% of state appropriations for community 
college operations in fiscal year 1996-97, dropped below 1% in 2001-02, stayed at that 
level until 2005-06, and then jumped to 1.8% (Dougherty & Natow, 2010). In addition to 
shifts in the amount of funds that are involved in performance funding programs, the 
particular performance indicators used to allocate those funds can also change. In Florida, 
the Performance-Based Budgeting program experienced considerable changes in the 
performance indicators used. Florida’s PBB added 10 performance indicators and 
dropped three in the 12 years between 1996-97 and 2007-08, an average of one change 
per year.^^ 



For an analysis of the fluctuations in funding and indicators in Florida, and how they contrasted with 
Tennessee, see Dougherty and Natow (2010). 
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7.3 The Brief Duration of Many PF Programs 

Many performance funding programs do not last for many years, thus 
undercutting their capacity to produce effects. At least half of all the states that have 
enacted performance funding programs have later abandoned them, often after only a few 
years. For example, performance funding lasted only one year in Minnesota, two years in 
Arkansas and Washington (in the 1990s), and four years in Illinois (Burke, 2002; 
Dougherty et ah, 2012). 

The instability of performance funding levels and indicators and the demise of 
many performance funding programs point to the political dynamics involved as 
performance funding programs are implemented and operate over time. Moneys for 
performance funding programs are affected by the waxing and waning of state revenues 
and by the fact that higher education budgets often serve as the place where state 
governments seek to balance their books in bad times (Callan, 2002; Zumeta, 201 1). In 
fact, state fiscal problems can result in the demise of performance funding programs. 
They often fail to get much support from higher education — particularly the state 
universities — when they are established. This lack of initial support can then turn to 
outright enmity — and the demise of performance funding — when fiscal pressures lead to 
declines in regular state funding for higher education. At that point, higher education 
institutions may push to eliminate performance funding in the name of defending their 
regular state funding (Dougherty & Natow, 2009; Dougherty et ah, 2012).^° Meanwhile, 
instability in performance indicators is rooted in part in pushback by colleges as they 
demand indicators that they regard as more appropriate to their missions and that will 
reward them better (Dougherty & Natow, 2010). 

7.4 Inadequate State Funding of Performance Funding 

A frequently cited reason for the lack of strong impacts of performance funding is 
the simple fact that not enough money is involved. First, the proportion of institutional 
budgets typically tied to performance funding is not enough to make a big enough 
difference in institutional attention (Dougherty & Hong, 2006; Shin, 2010). In several 

This becomes even more likely if the original champions of performance funding — such as certain key 
legislators or officials of the state higher education coordinating board — have left office due to retirement, 
term limits, and so on. (Dougherty & Natow, 2009; Dougherty et ah, 2012). 
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states with performance funding, the proportion of state funding of higher education that 
is tied to performance outcomes is 1% or less (Dougherty et ah, 201 1). This already weak 
effect is exacerbated by other factors. 

Institutional allocations under performance funding do not necessarily track 
performance well, but may still be driven primarily by input factors such as enrollments. 
An analysis of Florida’s Performance-Based Budgeting program found that, whatever the 
public intent, performance funding allotments remained highly predicted by institutional 
enrollments (Wright et ah, 2002, p. 155). 

Third, the state share of higher education revenues has dropped over the last 30 
years (Kane, Orszag, & Gunter, 2003; Zumeta, 2010). As a result, even if performance 
funding were to represent a significant proportion of state funding, its effect is weakened 
by the fact that state funding was dropping in relative terms. 

Finally, in Florida, funding under its Workforce Development Education Fund 
(1997-02) did not rise as fast as improved performance or enrollments. As a result, even 
if an institution raised its performance, its performance funding might not increase and 
could even drop if other institutions raised their performance even more substantially 
(Bell, 2005, pp. 156-157; Dougherty & Hong, 2006, p. 72; Gray et ah, 2001, p. 29; 

Poisel, 1998, p. 94). 

7.5 Shortfalls in Regular State Funding 

This issue was raised in five studies on Tennessee and one on Washington. In 
Tennessee, the funds allocated under the regular enrollment-based formula have not kept 
pace with enrollment growth (Freeman, 2000, pp. 88-89; Hall, 2000, pp. 93-94; Latimer, 
2001, pp. 95-98; Lorber, 2001, p. 85; Shaw, 2000, p. 97). For example, according to its 
chief financial officer, Walters State Community College received only 89% of the base 
funding called for by the state’s regular funding formula for the 1999-00 academic year 
(Shaw, 2000, p. 97). As a result, performance funding is no longer bonus funding but 
instead is used to make up the shortfall in regular state funding. A dean at the University 
of Memphis stated that performance money “gets chewed up just trying to keep the ship 
afloat on a day to day basis” (Latimer, 2001, p. 95). This practice eventually led a number 
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of Tennessee higher education officials to argue that performance funding provided the 
state with an excuse to cut the formula funding. 

Meanwhile, in Washington, even as the Student Achievement Initiative took 
effect in recent years, state formula funding dropped. As a result, many community 
college presidents and senior administrators became resentful, feeling that performance 
funding was no longer a bonus but rather only a partial redress of dropping state support 
(Jenkins et ah, 2009, p. 40; Shulock & Jenkins, 2011, p. 12). 

7.6 Uneven Knowledge About and Responsibility for Performance Funding 

The effective implementation of performance funding has also been hampered by 
the fact that awareness of performance funding and its requirements varies greatly within 
institutions, with those at the top of the hierarchy possessing greater understanding of and 
responsibility for the performance funding process than middle-level administrators and 
faculty who also play an important role in implementing performance funding. For 
example, in a survey of two-year and four-year college administrators in five states with 
performance funding, Burke (2002, p. 63-64) found that while 88% of the top 
administrators were “very familiar” or “familiar” with their state’s performance funding 
program, only 58% of the academic deans and 40% of the department chairs were 
familiar with it. This finding also shows up in seven studies on Tennessee and in one 
each on North Carolina, Ohio, and Washington. 

In Ohio, a survey of 224 administrators at 13 public universities revealed that 
knowledge of performance funding was stratified within institutions in a manner similar 
to that described by Burke (2002). Executive-level administrators such as presidents and 
vice presidents were more knowledgeable than were department/unit-level 
administrators, with 38% of the former but only 22% of the latter reporting that they were 
aware of the state’s Success Challenge performance funding program (Schaller, 2004, p. 
151 ).^^ 

In Washington, interviews at 17 community colleges established that, while the 
state’s Student Achievement Initiative (SAI) performance funding program was known 

Schaller also reports that student service administrators apparently were more knowledgeable about the 
Success Challenge than were academic administrators, but the difference was not statistically significant 
(Schaller, 2004, pp. 86, 89, 92). 
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and understood “fairly well” to “very well” by presidents, senior administrators, and 
institutional research staff, the same was not true of faculty and student support services 
staff. In fact, the majority of faculty members interviewed had only a limited 
understanding of the SAI (Jenkins et ah, 2009, pp. 19-22, 33). The following description 
by a vice president of instruction at a Washington community college was typical: 

With our faculty we’ve told them that this initiative is 
happening. . . . Faculty know that something is happening, 
but that is the extent of it. ... The faculty have had it 
explained to them, but if you talked to them, they couldn’t 
explain it back. (As quoted in Jenkins et ah, 2009, p. 20) 

Similar inequality of knowledge about performance funding was also reported in 
Tennessee (Freeman, 2000, pp. 81-82; Hall, 2000, p. 73; Latimer, 2001, pp. 72-76; 
Shaw, 2000, pp. 66-67, 72, 86-87). At the University of Tennessee, Knoxville, an 
administrator noted: “I don’t think most people inside the university understand [the state 
performance funding system]. I would say 95% of the faculty don’t know anything about 
it” (as quoted in Hall, 2000, p. 74). 

This informational inequality has been attributed to a number of different causes. 
First, many administrators view performance data collection and analysis as an 
administrative task that faculty need not be concerned about (Freeman, 2000, pp. 81-84; 
Hall, 2000, p. 73; Harbour & Nagy, 2005, p. 453; Jenkins et ah, 2009, p. 21). For 
example, at Volunteer State Community College in Tennessee, a senior administrator 
noted: 



[Faculty] don’t need to know. To me our campuses now are 
large enough, and they’re diverse enough, and they’re so 
specialized that people . . . really don’t have the time, 
energy, or intellect . . . everybody to become an expert on 
the aspects of performance funding. (As quoted in 
Freeman, 2000, p. 82) 

In Washington, administrative reluctance to widely publicize performance funding within 
their institutions was tied to uncertainly about its longevity and implications. College 
administrators were leery about widely publicizing the performance funding program 
until they got a better idea of how it would work and whether it would last. They 



35 




reportedly did not want to involve faculty in an evanescent effort that they might well 
resist (Jenkins et ah, 2009, p. 21). 

Further, lack of faculty awareness also may be tied to a faculty perception that 
performance funding is not central to the faculty role (Jenkins et ah, 2009, p. 21; Shaw, 
2000, p. 87). A faculty member at Walters State Community College in Tennessee noted: 

As a faculty member . . . most of your work is wrapped up 
in your discipline, preparing notes for class, and spending 
time with students. Only when you as a faculty member are 
forced to address those issues regarding performance 
funding, do you participate and integrate them. (As quoted 
in Shaw, 2000, p. 87) 

In Washington, a factor contributing to faculty perceptions that performance funding is 
not very relevant to their jobs is the fact that most of the community colleges have 
focused their efforts initially on student services and improving basic skills, which are 
emphases at the margins of effort and attention of most college-level faculty (Jenkins et 
ah, 2009, p. 21). 

Another factor contributing to lack of knowledge and feeling of responsibility on 
the part of faculty and middle-level administrators is the fact that performance indicators 
are typically measured at the institutional level alone and not at the unit level as well. As 
a result, faculty and middle-level administrators may not be aware of the performance of 
their particular academic or administrative units relative to other units at their college or 
comparable units at other colleges (El-Khawas, 1998, p. 325; Ewell, 1994).^^ 

Buttressing this lack of awareness and responsibility at the unit level is the fact 
that performance funding typically flows into the general operating funds of institutions. 
Allocating performance funds to the general operating fund makes it difficult for those 
not directly responsible for the overall institutional budget to see the connection between 
their actions and the receipt of performance funding (Ereeman, 2000, p. 93; Hall, 2000, p. 
92; Eorber, 2001, pp. 84-85). Eor example, a department chair at Tennessee Technical 
University argued: “[Money from performance funding] is for the general fund and to 



This may be quite deliberate, as a way to shield the technical core of the college from outside inspection 
(Meyer & Rowan, 1977/1991; see also El-Khawas, 1998, p. 326). 
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most faculty that’s a black hole. . . . What am I going to get out of this? Nothing. . . . Do I 
get travel? No. Do I get a new computer? No” (as quoted in Lorber, 2001, p. 85). 

Lack of faculty knowledge about and perceived responsibility for performance 
funding makes it hard to mobilize faculty efforts to make it effective. College success in 
meeting performance demands cannot be done only through administrative action but 
must ultimately involve the concerted action of the faculty, which in turn requires their 
knowledge and acceptance of performance funding. Moreover, lack of in-depth 
involvement by faculty and mid-level administrators in the design and implementation of 
performance funding programs raises the possibility of unintended impacts that 
administrators and others cannot anticipate. 

7.7 Inequality of Institutional Capacity 

Two studies on Florida and one on Washington discuss how differences in 
institutional capacity are an obstacle to effective implementation of performance funding 
(Bell, 2005, p. 135; Dougherty & Hong, 2006, p. 73; Jenkins et ah, 2009, p. 28). In 
Florida, an official of the Florida state community college system noted: 

Florida has spent a lot of money trying to get all 28 of its 
institutions with good technical systems and good computer 
systems. But some of our smaller colleges . . . where there’s 
just so few people in some of these [institutional research] 
programs, I know this is causing them a lot of problems 
trying to keep up with things. (As quoted in Dougherty & 

Hong, 2006, p. 73) 

Meanwhile, in Washington, an evaluation of the recently established Student 
Achievement Initiative performance funding program for community colleges revealed 
wide disparities in institutional capacity to collect and analyze performance data. The 
data supplied by the Washington State Board for Community and Technical Colleges to 
the community colleges need to be supplemented by data collected by the institutions 
themselves, but the colleges differ widely in their capacity for data analysis. At several 
colleges, there is a shortage of institutional research (IR) staff with the skills and time to 
rigorously analyze college performance data. And even colleges with larger IR 
departments still have to collect and analyze their own data, and they differ widely in 
their capacity to do so (Jenkins et ah, 2009, p. 28). 
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7.8 Institutional Resistance to and Gaming of the System 

The obstacles to the effective implementation of performance funding are matters 
not just of capacity but also of will. Several studies on Tennessee, and one each on 
Florida, Missouri, Ohio, South Carolina, and Washington, document ways in which 
institutions try to game the performance funding program to secure high performance 
scores without actually improving their performance. This gaming takes two main forms: 
setting low institutional goals that can be easily attained and taking actions that produce 
apparently desirable performance but in ways that require minimal effort and are not in 
keeping with the spirit of performance funding. 

Setting low goals. One form of gaming occurs in systems that allow institutions 
to set their own goals or targets. Institutions can set goals that are easily achievable rather 
than goals that stretch the institution. For example, in Tennessee, in order to ensure they 
would receive full points on the performance indicators, some institutions set artificially 
low goals that they were sure to clear (Freeman, 2000, pp. 89-90; Latimer, 2001, pp. 90- 
91; Williams, 2005, p. 94). Similar behavior seemingly occurred as well in South 
Carolina (South Carolina Legislative Audit Council, 2001, p. 19). 

Deceptive compliance. In some instances, institutions have complied with the 
requirements associated with performance funding but only minimally and deceptively. 
Given budget concerns and the potential for performance funds to be directed towards 
general operations, administrators have looked for ways in which performance points can 
be increased without substantial expenditures, effort, or even actual improvements. Three 
studies on Tennessee and two on Florida discuss ways in which participants have tried to 
game the system. 

A vice president at a Tennessee university described how programs can 
manipulate their student assessment results by postponing field exams that are likely to 
yield lower results because “you don’t want a low score to affect you for five years” 
(Lorber, 2001, p. 72). Also, a faculty leader at one Tennessee university noted that 
departments could secure favorable external reviews of their departments by calling on 
friends to perform the external audits (Baxter, Brant, & Forster, 2008, p. 58). Meanwhile, 
in Florida, a faculty member at a community college stated that he was encouraged to 
advise students to take a full sequence of courses even if they did not need several of the 
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courses because the college would get extra performance points if students successfully 
completed course sequences (Bell, 2005, p. 121). A more palatable version of this policy 
was followed by a community college that gave college credit to vocational certificates, 
with the hope that it would induce students to go on to get an associate degree, for which 
the college could get extra performance points (Morris, 2002, p. 174). Finally, in Ohio, a 
number of university branch campuses were found to have relabeled as transfer students 
rising juniors who remained within the university system, in order to fulfill the transfer 
expectation of the Performance Challenge (Dunlop-Loach, 2000, p. 92). 



8. Unintended Impacts of Performance Funding 

Policymakers announce certain goals when adopting performance funding, but as 
with any policy intervention, there are also consequences associated with performance 
funding that are unintended, at least publicly (Dougherty & Hong, 2006, p. 73). The 
unintended impacts uncovered in our review of the research literature include costs of 
compliance, narrowing of institutional missions, restriction of student admissions, and 
grade inflation and weakening of academic standards. 

8.1 Compliance Costs 

Four studies on Tennessee, two each on Florida and South Carolina, and one on 
Missouri discuss the costs, in money and time, of complying with performance funding 
mandates. The financial costs are tied to the need for additional personnel to handle the 
data collection and reporting required by the state performance funding program 
(Dougherty & Hong, 2006, p. 76). Walters State Community College in Tennessee had to 
expand its Office of Planning, Research, and Assessment in order to gather all the data 
required by the state (Shaw, 2000, pp. 76-77; see also Hall, 2000, p. 93; Latimer, 2001, 
p. 79; Williams, 2005, p. 92). A Florida administrator described the state’s data demands 
as an “administrative nightmare,” noting that local data had to be recoded to match state 
databases and that it was often difficult to match local job titles with the state database 
(Gray et ah, 2001, pp. 31-32, 36). 
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A major hole in the available research on performance funding is the lack of good 
studies on the cost to institutions of meeting state demands for performance data, 
developing effective organizational learning capacity, mounting initiatives to improve 
institutional performance, and evaluating the results of those initiatives. Better cost 
estimates would be very important to determining whether the costs to institutions of 
performance funding outweigh the fiscal benefits and therefore whether states need to 
make to concerted efforts to offset those costs if they wish performance funding to be 
welcomed by colleges. 

8.2 Narrowing of Institutional Missions 

Two Florida studies, as well as single studies of North Carolina and Washington, 
discuss how performance funding can lead to a narrowing of institutional missions. 
Colleges may deemphasize missions that are not rewarded or only minimally rewarded 
by the performance funding program. 

One mission that may suffer is transfer education. In Washington, personnel at six 
of the 17 community colleges expressed concern that the state’s Student Achievement 
Initiative might disadvantage institutions with a focus on academic transfer, since the 
initiative emphasized improvement in basic skills and developmental education and did 
not include an indicator for successful transfer (Jenkins et ah, 2009, pp. 36-37).^^ 

Another mission that may suffer is general education (Dougherty & Hong, 2006, 
p. 76). Valencia Community College in Florida instituted a “60 hour rule,” stating that 
programs could not exceed 60 credit hours. The hope was to speed up time-to-completion, 
but the side effect was to limit students’ ability to take more electives and get a more 
general education (Bell, 2005, pp. 117-118). Furthermore, in Washington, some 
community college respondents felt that the Student Achievement Initiative might lead to 
a shift of focus and resources away from college-level coursework. A faculty member 
noted: 

How much do we fund developmental education at the 
community college, and when [do] you start transferring 
those resources into developmental education? Does that 



It could be argued, however, that the transfer function of the Washington community colleges was aided 
indirectly by the SAI performance indicator for completing college level mathematics. 
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mean we will be losing our calculus classes? Do you cut 
back on literature courses and offer more sections of 
developmental education? I mean, are we a college at that 
point? (As quoted in Jenkins et ak, 2009, p. 36) 

Performance funding can also narrow the workforce training mission of 
community colleges. In Florida, because job placement points were awarded only for jobs 
with wages in excess of $9 an hour, programs that led to jobs in fields with lower wages 
were terminated, even if those programs served local interests, like nurse-assistants or 
childcare providers (Bell, 2005, p. 102; Gray et ah, 2001, p. 40). 

8.3 Grade Inflation and Weakening of Academic Standards 

Three studies on Tennessee and one each on Florida and Washington discuss how 
performance funding leads to grade inflation and the weakening of academic standards. 

In Florida, several community colleges removed various obstacles to their students’ 
degree completion. Often these obstacles were simply unnecessary hindrances, but the 
obstacle clearing could also result in the elimination of difficult, but important, 
intellectual requirements (Dougherty & Hong, 2006, pp. 73-74). There is also some 
evidence that colleges are being pushed by accountability demands for higher retention 
and graduation rates to pressure faculty to avoid giving failing grades. The president of 
the American Association of University Professors chapter at a Florida community 
college noted: “There’s a lot of pressure to retain every single student no matter what it 
takes. . . . We have to report every conference we’ve had, the outcome, if the student 
wasn’t retained, why, how many efforts were made” (as quoted in Dougherty & Hong, 
2006, pp. 74-75). These faculty fears also crop up in the studies of Tennessee and 
Washington (Banta et ah, 1996, p. 36; Freeman, 2000, p. 90; Jenkins et ah, 2009, p. 39; 
Tanner, 2005, p. 84). In Washington, faculty and administrators at several community 
colleges raised the alarm that the Student Achievement Initiative might lead to pressure 
on instructors to lower their academic standards so that more students would pass courses 
and the colleges could gain more performance points (Jenkins et ah, 2009, p. 39). 
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8.4 Restriction of Student Admissions 



Two studies on Florida and one on Missouri note how performance funding can 
lead colleges to restrict the admission of less prepared students in order to boost their 
retention and graduation rates, an example of what has been called “creaming.” In 
Florida, a local community college official noted: 

There are people who may need to take a course in a 
program and we would not necessarily want to attract those 
people because you’re going to be working for 
performance-based funding ... in the health sciences, this is 
a major concern because it’s not who you start with, it’s 
who completes that matters. (As quoted in Dougherty & 

Hong, 2006, p. 75) 

In fact, another study found that a Florida community college had restricted enrollments 
to maintain program quality. Moreover, that college had eliminated its Center for 
Disabilities because its completion and job placement rates did not justify the high cost of 

24 

running the program (Bell, 2005, p. 146). 

8.5 Diminished Faculty Voice in Academic Governance 

The low faculty awareness of performance funding that we discussed above also 
carries an important publicly unintended impact: a diminished faculty voice in academic 
governance. If faculty members are often unaware of the extent and content of 
performance funding, they are less able to shape how colleges respond to it. This not only 
undercuts the role of faculty in the shared governance of institutions but also increases 
the possibility that the performance funding programs may be designed and implemented 
in ways that are less effective and could produce unintended impacts due to a lack of 
awareness of the particulars of instruction and administrative practice that faculty and 
midlevel administrators are privy to. 

Now and again the studies we reviewed included reports of some faculty 
unhappiness with the relatively peripheral role faculty played. For example, a faculty 



For community colleges, becoming more selective in student intake is more difficult to do — practically 
and ideologically — than it is for four-year colleges, but it is by no means impossible. Community colleges 
can downplay outreach efforts to high schools that have higher numbers of disadvantaged students. They 
can also cut back on their offerings of English as a Second Language or developmental education or adult 
basic education. 



42 




member at Tennessee Technological University argued: “To be expedient, they [upper 
administration] sometimes do certain things (e.g., set achievement goals) without what I 
would consider appropriate discussion or consultation” (as quoted in Lorber, 2001, p. 78). 
More pointedly, a faculty leader at Volunteer State Community College observed: 

When [an executive administrator] first came to campus . . . 

I was chairing a committee on faculty workload at that time. 

... He said, you do not need to worry about performance 
funding. ... I think that sometimes there is that sort of 
attitude on this campus. We will let the faculty do their 
thing and the other group of administration will make the 
decisions. (As quoted in Freeman, 2000, p. 84) 

Unfortunately, the studies we reviewed did not explore to any great degree the impact of 
performance funding on institutional governance and the relative power of faculty and 
administrators. This is a topic that deserves more study. 



9. Summary and Conclusions 



9.1 Main Findings 

The studies reviewed suggest that tying funding to outputs has the ability to 
modify institutional behavior. They provide evidence that performance funding does have 
immediate impacts on colleges in the form of changes in funding, greater awareness of 
state priorities and of their own institutional performance, and increased status 
competition among institutions. Furthermore, there is evidence that performance funding 
does lead to intermediate changes in the form of greater use of data in institutional 
planning and policymaking and changes in academic and student service policies and 
practices intended to improve student outcomes. 

At the same time, the research literature does not provide firm evidence that 
performance funding significantly increases rates of remedial completion, retention, and 
graduation. When these claims are made, they are not based on solid data that control for 
other possible causes of changes in student outcomes beyond performance funding. In 
fact, the few multivariate quantitative analyses of the impacts of performance funding on 
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institutional retention and graduation rates uniformly fail to find statistically significant 
positive impacts. 

The research literature on state performance funding documents many different 
obstacles to its effective functioning. Among them are inappropriate measures of 
institutional performance, performance funding that lags behind enrollment growth, 
shortfalls in regular state funding for higher education, the early demise of many 
performance funding programs, uneven knowledge about performance funding within 
institutions, inequality of institutional capacity, and institutional resistance and gaming of 
the performance funding system. 

The research literature also documents various unintended impacts of 
performance funding. They include costs of compliance, narrowing of institutional 
missions, grade inflation and lowering of academic standards, restrictions on student 
admissions, and diminished faculty voice in academic governance. 

9.2 Limitations of the Research Literature and How to Address Them 

The research literature on the impacts of performance funding is instructive in 
many ways but it is also marked by several limitations. First, there are too few studies 
that examine the impacts of performance funding using multivariate quantitative 
methodologies (Fryar, 2011; Rabovsky, 2011; Sanford & Hunter, 2011; Shin, 2010; and 
Volkwein & Tandberg, 2008), and the quantitative studies that are available do have a 
number of limitations. All examine the impacts on student outcomes at four-year 
institutions, leaving out community colleges. In addition, the independent variable for 
performance funding presence typically only measures whether or not a state has had a 
performance funding program.^^ However, it is important to also take into account 
various features of the performance funding program that may make a difference: how 



If these studies were to include community colleges, it would be important that quantitative studies 
include as a dependent variable a measure of graduation and transfer combined, in order not to penalize 
community colleges that have many students who successfully transfer but who do so without an associates 
degree. 

This alone is not easy to determine. Surveys confront the fact that state officials not infrequently disagree 
in their understanding of what performance funding is and whether their state has it. As a result, a 
respondent to a survey may claim that their state does have performance funding, but this claim may not be 
validated by an investigation of the documentary record and interviews with other actors in the state. 
Moreover, some states have enacted performance funding and then later suspended funding for it, but still 
hold that performance funding remains in place. 
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long has the program been in place; what proportion of state appropriations for higher 
education has it accounted for (which often varies over time); how is the performance 
funding allocated across various outcome measures (which again typically varies over 
time); and whether performance funding has been flanked by other state performance 
accountability programs (such as performance reporting and performance budgeting) that 
may amplify or dissipate its impact. Finally, the studies could use more control 
variables. It is important to include a wide range of both institutional and state 
characteristics variables. In the case of institutional variables, it would be important to 
control for institutional size (enrollments), student composition (socioeconomic status, 
race/ethnicity, gender, and part-time or full-time status), institutional selectivity, 
institutional mission (e.g., Carnegie classification), revenues from tuition, expenditures 
per FTE, expenditures for instruction, academic support, and administration, percentage 
of the faculty that is part time, and the degree to which the institution is residential. With 
regard to state characteristics, it would be important to control for private sector 
enrollments, state per capita appropriations for higher education and for other than higher 
education, state aid as a proportion of state appropriations for higher education, state 
spending on need based aid, and region of the country, among other variables. 

Second, the research literature on performance funding is restricted in the forms 
of performance funding it analyzes. It is focused on the traditional forms performance 
funding has taken: allocating funds that are in addition to regular state funding for higher 
education on the basis of indicators such as rates of retention, graduation, and job 
placement (performance funding 1.0). There is a great need for studies that examine 
newer forms of performance funding, or what has been dubbed PF 2.0: programs that use 
intermediate measures (such as passing gatekeeper courses or reaching certain credit 
thresholds) and, even further, embed performance indicators in the basic state funding 



The Shin (2010) study does take into account some of these variable features of performance funding 
programs. An interesting possibility that should be pursued by further quantitative studies is whether the 
joint impact of having both performance funding and performance budgeting operating together is 
significantly greater than having one or the other alone (see Shin & Milton, 2004). 

All of these factors have been found to be statistically significant predictors of institutional graduation 
rates in one or another study of the impact of performance funding (Fryar, 2011; Sanford & Hunter, 2011; 
Shin, 2010; Volkwein & Tandberg, 2008) or in other studies examining predictors of graduation from 
community colleges or four-year colleges (Calcagno, Bailey, Jenkins, Kienzl, & Leinbach, 2008; Scott, 
Bailey, & Kienzl, 2006; Titus, 2006). 
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formula itself rather than having performance funding be a bonus. The recently 
developed performance funding programs in Ohio and Tennessee (as well as the Student 
Achievement Initiative in Washington) are notable examples of these new forms of 
performance funding that need to be carefully studied. In the case of Tennessee and Ohio, 
the fact that they have dispensed with enrollment-based funding for four-year colleges 
and (in Tennessee) the two-year colleges as well marks a sharp departure in the history of 
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funding for higher education. 

Third, the qualitative research literature on performance funding is restricted in 
the scope of the state contexts it examines. It is unusual to have studies that examine 
more than one state at a time, allowing analysis of the impact on institutional and student 
outcomes of state differences in the design and implementation of performance funding, 
the structure, governance, and missions of the higher education system, and the nature of 
the social, economic, and political systems. 

Fourth, the qualitative research literature is also restricted in the scope of 
institutional contexts it examines. Many qualitative studies investigate just one institution 
or, occasionally, a number of institutions of the same type, such as community colleges. 

It is rare to find studies that sample across different kinds of institutions, such as flagship 
state universities, regional public universities of different types, and community colleges 
of different types. As a result, there is little opportunity to determine how the 
implementation and impacts of performance funding differ across institutional types. 

Finally, the existing qualitative studies too often do not triangulate their data or do 
not do it carefully enough. There is too much reliance on simple self-reports, often from a 
rather limited number of interviewees within any given institution. As a result, it is often 
hard to gauge how well they are really capturing the true dimensions of institutional 
responses to performance funding. The self-reports rarely carry any clear metrics to allow 
accurate gauging of the breadth and depth of institutional responses to performance 
funding. Moreover, most studies make little effort to cross-check the accuracy of 
interviewee responses against institutional and state documentary data. 



Kevin Dougherty is leading a study, with support from Lumina Foundation, examining the impacts of the 
new PF 2.0 programs in Ohio and Tennessee on various kinds of colleges. For a description of the project, 
go to http://ccrc. tc.columbia.edu/Collection. asp?cid=74 



46 




9.3 Conclusions About the Ultimate Impacts of Performance Funding 

The absence of findings that performance funding does produce significant 
improvements in student outcomes should not lead us to dismiss it. The multivariate 
studies mentioned above are still too few in number to reach definitive conclusions. 
Moreover, they apply to the traditional form of performance funding (what has been 
dubbed performance funding 1.0), involving small bonuses to base state funding for 
higher education. They do not apply to new forms (performance funding 2.0) that embed 
performance indicators in the base state funding formula and involve much more money. 
Those new forms may have significant impacts, if only because they involve 
considerably greater funds. Finally, there are a host of obstacles to performance funding 
that, if removed, might greatly improve its effectiveness. 

In the next section, we address how those obstacles might be reduced and perhaps 
even eliminated. We also address the key issue of how to reduce the possibly quite 
significant unintended impacts of performance funding. If these unintended impacts were 
to become quite sizable, particularly under a more forceful performance funding regime, 
they would call into question the worth of performance funding as an educational policy. 

9.4 Ways to Address the Problems of Performance Funding 

Despite the limitations of the research literature reviewed above, the research 
clearly indicates that existing performance funding programs encounter sizable obstacles 
to success and produce significant unintended impacts on students and colleges. We thus 
need to know much more about how widespread those problems are, how they arise, and 
what is being done to solve them.^° In the meantime, however, many states are currently 
considering performance funding, and they need advice now on how best to combat the 
obstacles encountered and unintended impacts produced by performance funding 
programs. 



These topics are the subject of the research study mentioned in the previous footnote. The study is 
examining the implementation and impacts (intended and unintended) and obstacles to effectiveness of 
performance funding programs in three states (Florida, Ohio, and Tennessee). Within each state, different 
kinds of institutions will be examined, including research universities, other kinds of state four-year 
colleges, and community colleges. 
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Reducing obstacles to performance funding effectiveness. In order to reduce 
these obstacles, performance funding programs need to utilize better performance 
indicators and measures, provide more performance funding and insulate it better from 
the state revenue cycle, improve organizational capacity, and combat institutional 
resistance and gaming. 

Improving indicators and measures. States can address the criticism that 
performance funding systems often use indicators that do not accurately gauge the actual 
successes of colleges by using different measures of existing ones and adding other 
indicators of success. In the case of retention and graduation, graduation measures for 
community colleges should be broken down by whether or not students do intend to get a 
degree, a predisposition that could be determined by whether students have attempted a 
certain minimum number of credits within their first year of college (Bailey, 2011; 
Claggett, 2011; Committee on Measures of Student Success, 2011; Offenstein & 

Shulock, 2010).^^ 

In addition, community college graduation measures should be coupled with 
transfer measures. Given how many students transfer from a community college to a 
four-year college without first getting a community college degree, performance funding 
programs should include indicators of successful transfer and pair them with measures of 
graduation (Claggett, 2011; Committee on Measures of Student Success, 2011; 
Dougherty & Hong, 2006; Jobs for the Future, 2008).^^ 

Another key consideration in producing data that fairly represent community 
college outputs is extending the time frame for tracking outcomes for students. Many 
students attend community colleges part time or have to begin by taking noncredit 
remedial courses and therefore do not complete a degree or demonstrate other successful 
outcomes within the three years mandated by the federal Graduation Rate Survey. When 



For example, student intention to get a degree could be determined by whether students exhibit a 
combination of having taken more than six credits in the first year and enrolled within the first two years in 
a course (such as college math or English), a combination that is usually exhibited only by degree-seeking 
students (Offenstein & Shulock, 2010). 

This has also been recommended by the Committee on Measures of Student Success appointed by 
Secretary of Education Arne Duncan (Committee on Measures of Student Success, 2011; Nelson, 201 1). In 
order to capture all transfer students, researchers recommend that the National Student Clearinghouse be 
used, rather than relying just on reports from in-state public four-year colleges and universities (Bailey, 
2011; Offenstein & Shulock, 2010). 
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students are instead followed up six years after entry, completion rates go up sharply 
(Calcagno, Bailey, Jenkins, & Leinbach, 2008; Jobs for the Future, 2008; Offenstein & 
Shulock, 2010).^^ 

The difficulties of some colleges — particularly those in rural areas — in meeting 
demands for job placement in well-paying jobs can be addressed by performance funding 
programs that acknowledge local labor market differences. Corrections for geographical 
and temporal differences in labor market conditions should be built explicitly into job and 
wage placement measures. 

Similarly, corrections are needed to address the many colleges — particularly 
community colleges — that are open access institutions designed to enroll disadvantaged 
students who are less likely to persist and graduate. Without such corrections, colleges 
that enroll many disadvantaged students or students with weak degree aspirations will be 
penalized for enrolling such students. We will return to this point below. 

Providing more performance funding. The small size of most performance 
funding programs and the lack of significant impacts on ultimate outcomes argues for 
increasing the proportion of state funding that is premised on performance outcomes. 
However, there is no clear evidence on the optimal level of funding. The fact that even 
Tennessee’s program, with nearly 5% of state funding being based on performance, 
appears not to have had significant impacts on graduation and retention rates (Sanford & 
Hunter, 2011) argues that the percentage has to be even higher, perhaps much higher. 
However, this inference has to be balanced with awareness that programs that allocate 
much higher proportions of state funding on the basis of performance indicators may 
engender a powerful backlash or produce significant unintended impacts. 



At the same time, it is not certain that longer follow-ups will produce a radically different picture of 
which institutions are doing better and which worse. It may be that the colleges with the best outcomes 
when students are followed up three years later will also be at the head of the pack when the follow-up is 
after six years. 

However, we should not assume that most community college entrants do not want a degree. The 1996 
Beginning Postsecondary Student survey found that 90% of first-time community college entrants wanted 
at least a certificate, with 63% wanting at least a baccalaureate degree (Hoachlander, Sikora, & Horn, 2003, 
pp. 10-1 1). Even if this were not the case, community colleges should still have as one of their central 
missions strengthening the aspirations of less advantaged students. Redesigned retention and graduation 
measures must not let weak credential motivation on the part of many community college entrants become 
an excuse for tolerating high dropout rates. One way of preventing such a practice is to gradually increase 
retention and graduation targets from year to year, spurring colleges to “warm up” their students’ 
aspirations. 
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State performance funding programs can address the frequent criticism that 
funding for those programs often fails to keep up with rising enrollments and improved 
performance by indexing state performance funding to both. However, this is not an easy 
task, given the instability of state revenues over time and the typical resort to cuts in 
higher education appropriations as a means of coping. This leads to our next point. 

Insulating performance funding from the state revenue cycle. In order to 
maintain relatively stable funding or even to survive, performance funding programs need 
to be insulated from the ups and downs of the state revenue cycle. This is hard if 
performance funding takes the form of a bonus on top of regular state funding of higher 
education. Hence, it has been proposed that states embed performance indicators in the 
base state funding formula for higher education, as has been done in Tennessee, Ohio, 
and Indiana and proposed in a number of other states (Snyder, 201 1). In this case, 
performance funding will still drop if state funding for higher education declines, but it 
will be less subject to being eliminated because it will not take the form of a separate 
program that colleges will want to sacrifice first before enduring cuts to their regular, 
formula-based state funding (Dougherty et ah, 2012). 

Improving institutional capacity. To help colleges that have fewer fiscal and 
human resources, state performance funding programs need to help them to improve their 
capacity to create a culture of inquiry and engage in systematic organizational learning 
(Argyris & Schon, 1996; Bensimon, 2005; Dowd & Tong, 2007; Jenkins, 2011; Kerrigan, 
2010). At the very least, they will need funds to acquire new data management systems 
and train institutional research staff and faculty to do research. Moreover, resource-poor 
colleges need assistance in developing their capacity to devise solutions to performance 
problems. This entails providing technical assistance and creating opportunities for 
colleges to create communities of practice with colleges facing similar challenges (Dowd 
& Tong, 2007; Shulock & Jenkins, 2011). Otherwise, a performance funding program 
may create a vicious cycle in which resource-poor colleges generate poor performance, 
which leads to declining state aid, which in turn further weakens their institutional 
capacity. 

More generally, colleges need help becoming learning organizations. They need 
to become capable of continually monitoring their performance, identifying problems. 
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devising strategies to resolve them, and evaluating how well those strategies work 
(Jenkins, 2011; Kerrigan, 2010). In this effort, there is mueh to be learned from the 
literature on organizational learning (Argyris & Schon, 1996; Huber, 1991; Lipshitz, 
Popper, & Friedman, 2002; Yorks & Marsiek, 2000). 

In cases where states are newly adopting performance funding, colleges can be 
assisted in getting acclimated by emulating Washington State in providing for a “learning 
year.” In such a year, colleges are informed of their performance results and assisted in 
developing responses to performance difficulties, but their performance does not yet 
affect their funding (Jenkins et ah, 2009; Shulock & Jenkins, 2011, p. 13). 

Lessening resistance and gaming. Finally, to reduce the incidence of 
institutional gaming of the performance funding system, states need to indicate what 
kinds of college efforts constitute valid responses to state performance funding demands 
and to inspect more carefully how colleges respond. However, it is important to keep in 
mind that one of the most important bulwarks to gaming the system is colleges’ 
allegiance to the goals of performance funding to begin with. Fostering this allegiance 
requires bringing colleges into the design of the system so that they see it as embodying 
higher education ideals and professional input and not as a top-down intrusion into the 
body collegiate. The lack of a widespread faculty awareness of and perceived 
responsibility for performance funding can be remedied by keeping such funding separate 
from the general revenues of colleges. The funding could be assigned to a dedicated 
account and spent on projects clearly designed to further improve student outcomes. 
Faculty awareness and buy-in could be enhanced by encouraging projects that are 
faculty-initiated and -run (see Shulock & Jenkins, 2011 for how this occurred at some 
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Washington State community colleges). 

Reducing or eliminating unintended negative impacts. Even if performance 
funding programs can be made more effective, the research also indicates that they 
produce a range of unintended impacts that need to be actively mitigated and even 
eliminated. Below are recommendations about how this could be done. 



The institutional resistance and gaming can also involve lowering academic standards. For solutions to 
this, see below. 
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Lowering compliance costs. Compliance costs — particularly for underresourced 
colleges — can be met by providing dedicated funding to underwrite compliance efforts. 
Also, states need to be careful to try to minimize their data demands on colleges, relying 
as much as possible on data that the colleges are already collecting for other purposes 
(Dougherty et ah, 2009). 

Combating narrowing of institutional mission. The danger of a narrowing of the 
colleges’ missions can be combated by specifying indicators addressing all important 
missions. States should emulate Ohio, Tennessee, and Washington State by using success 
in remedial education as a performance indicator, at least for community colleges 
(Jenkins et ah, 2009; Ohio Association of Community Colleges, 2010; Tennessee Higher 
Education Commission, 2011; Washington State Board for Community and Technical 
Colleges, 2007). Moreover, states need to also include measures pertaining to general 
education and continuing education, important community college missions that have 
been largely ignored by performance funding programs and thus face the danger of being 
neglected by institutional leaders. 

Protecting academic standards. The pressure on colleges to resort to grade 
inflation and lower standards in order to retain and graduate students is not easily 
relieved. However, states and colleges can carefully monitor degree requirements and 
course grade distributions to see if they have changed substantially after the advent of 
performance funding. Moreover, they can conduct anonymous surveys of faculty to see if 
they report substantial pressure to weaken academic requirements in order to keep up 
rates of retention, course completion, and graduation. Finally, states can conduct 
learning assessments. While indicators for specific kinds of learning are important, 
consideration should also be given to general student learning (Dougherty et ah, 2009).^* 



This recommendation exists in some tension with the recommendation, also valid, that states should 
avoid having so many performance indicators that they overly fragment institutional attention (Shulock & 
Jenkins, 2011). 

Both kinds of data are being collected as part of the research study by Kevin Dougherty and colleagues 
on the impacts of performance funding on colleges (Dougherty, 201 1). 

Indicators of general learning provide colleges with an incentive to foster students’ general education and 
avoid watering down curricular requirements or grading standards. The development of student learning 
indicators and measures should involve faculty in order to increase the probability of securing indicators 
and measures that are seen as legitimate and instructionally valid. 
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Avoiding disincentives to enrolling disadvantaged students. The temptation to 
improve institutional performance by reducing the admission of less prepared students 
(who are typically less advantaged socioeconomically) can be reduced by allowing 
performance targets to vary across colleges according to their student characteristics, by 
comparing colleges to peer colleges, or (as is done by the Student Achievement Initiative 
in Washington) by comparing a college’s performance now to its performance in the past 
(Bailey, 2011; Shulock & Jenkins, 2011, p. 7; Zumeta, 2001, p. 187).^^ 

However, it is important to also consider direct incentives to colleges for 
admitting less advantaged students who are less likely to persist in and graduate from 
college, including low-income, minority, and older students (Dowd, 2003, pp. 109-116). 
For example, under the new performance funding program in Tennessee, institutions are 
eligible for a 40% bonus for credit and degree completion by low-income and adult 
students (Tennessee Higher Education Commission, 2011). Moreover, Ohio’s new 
system of performance funding gives greater weight to completions by students at risk, 
defined initially in terms of being eligible for state need-based aid but with the prospect 
that other social background variables will be taken into account as well (Ohio Board of 
Regents, n.d.; Petrick, 2010). 

The importance of extensive consultation with institutions. A prerequisite for 
many of the actions recommended above is extensive consultation with higher education 
institutions. Such consultation is important to securing optimal results from performance 
funding programs. College administrators and faculty can help identify performance 
indicators and measures and funding practices that may produce deceptive results or 
perverse impacts. They can also help identify what kinds of assistance colleges need in 
order to be able to make the kinds of intermediate institutional changes required to 
produce the ultimate performance outcomes state officials seek (Jenkins, 2011; Shulock 
& Jenkins, 2011, pp. 13-14). But if nothing else, extensive consultation is important in 
protecting performance funding from being eliminated in periods of fiscal stress when 

Regression analysis can be used to develop predicted performance targets for colleges based on their 
student composition. However, such a system carries the danger of enshrining already existing selective 
admissions practices on the part of colleges (see Bailey, 1988, for a study of this issue with regard to the 
Job Training Partnership Act). 

^ See Coburn (2003), Coburn and Stein (2006), and Spillane, Reiser, and Reimer (2002) on the importance 
of faculty buy-in in the case of performance accountability regimes in K-12 schooling. 
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institutional political support is important (Dougherty & Natow, 2009; Dougherty et ah, 

2012 ). 
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